DOCIMEHT RBSUMB 



ED 071 .672 



LI 0011 067 



AUTHOR 
TITLE 

INSTITUTION 

SPONS AGENCY 

REPORT NO 
PUB DATE 
NOTE 

AVAILABLE FROM 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Clague, P* 

INSPEC SDI Investigation^ 1967-1969* Voliune I* 
Institution of Electrical Engineers, London 
(England) • 

Office for Scientific and Technical Information, 

London (England) • 

INSPEC-R~71-6 

71 

145pi; (0 References) ; Chapters 1 to It 

INSPEC^ The Institution of Elec'-rical Engineers, 

Savoy Place, London, HC2R OBL, Lngland (5 volumes^ HC 

$30*86) 

MF-$0«65 HC Not Available from EDRS*^ 
Economics; Electronics; Foreign Countries; ^ 
♦Information Dissemination; ♦Information Services; 
♦International Programs; Physics; ♦Relevance 
(Information Retrieval) ; ♦Use Studies 
England; INSPEC; Scientific and Technical 
Information; SDI; ♦Selective Dissemination of 
Information 



ABSTRACT 

The performance, economics and acceptability to users 
of the INSPEC SDI system were investigated in this study* The effect 
that provision of an SDI service might have on the information-use 
habits of users was also studied. The sauple group of users consisted 
of 600 individuals of whom 540 were drawn equally from: universities 
and colleges, government research establishments, and industrial 
firms* The remaining "60 were small groups of people having common 
information requirements. A corresponding sample of 600 people was 
used as a control group. The SDI operation was designed to provide a 
weekly service to users. All of the methods used to operate the SDI 
system and the problems encountered are discussed in some detail. The 
users comments on the service are also included* (Volumes II through 
V ares 'LI004068 through 004071*) (Author/NH) 
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AIMS AND nACK(?UOiyNI) OK TlflC rNSJ>f'X SDI .INVKSTIGAT ION 



Aims 

Tho aims of llic Tn v<^s I L^^a t i on W(*r<^ to study tlio po i^i orinanc<s 
ocoiiomKcs and accf^) tab i li ly to us(m\s o T th<^ SJXl. systoin and 
to attempt to discov(M^ the orfect that provision o an SDI 
S(*rvico m±(:Ut liavc^ on .infonmitlon - uso Jiabits o users. 

Rack^^roun(] 

The orl/yinaL j)roposal Vur the Xn v<;s tl^^a tion was put rorwax^d 
by the National R I eo t i^on i.cs K<isoaroh CounoJ I (nKI^C) to the 
OfTicci for ScLf^ntiri.c and Technical Inrorniation (oSTf:). This 
proposal ofivLsat^od a pJHJcr?i">nK^ oi' work dj vad<^d into six pJuises. 
Tho first two phas(^s oovcrin^v tho inanuaL f (*as i.bl I i. ty study and 
tho system d(*sj./.^„ was carried out by N.E.R^C. with the support 
of an OSTI ^rant and th<^ results of this work w(m.^<.- published 
in NERC report no; SDl/l dated Marcii 1966. 

Subsequently t}ie work wfis transf <^rj^(Hl to th<^ I jis titu tion of 
Eloctrlc^tl Enffin<M^rs from January 1967 to Vox^ui part of its 
newly- formcid INS PEC operation. 

fWith some mod i. Pica tions , mainly iji respect o the size oV th<* 
sample user and control, populations, the proposal for the 
remainin^r phases (Nos. 3-6) of th(^ Invos ti na t i on was re-submitted 
to OSTI and a /;rant was awarded to the Ins ti tui i oji to carry 
oujl this vor^k. 
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Phasps ')-6 coiii|)rj..s<»(l (Jic Co I Lowln^:- 

Phasc 3 Collc^clLon of sainpio of pari j.c i pan Is , comp.Lla- 
tion oV search profiles anil initial tostin^j of 
the system and the profili-^M. 

Phase ^1 ExporJ!n(Mi<;aL sorvico to users, profile analysis 
in lh(^ l.i/^ht oV exjx^r.i monta I rvsuLts and foed- 
bark l^roin uscirs* Ad justiiK^n t o I' profiles for 
start oV opcii^ationa I. service. 

Phase ^ Prtivis.ion of wcMikly (}p(^r\-i tiona 1 sorvico to 

nsc^rs, assessinonl oP S(»rv ic(» pc^rPoriiianco and 
aeeciptahi l.ily to usoi\s. 

Phase 6 Dotal l(^d analysis ol' rt^su Ms and wrilin/j; oP 
report • 

The planned duration of Phases 3 and h was six iiioiitlis (s-ich 
and of Phase 5 tvc^vci months. A proposed chiration of -six 
montlis for Phuso 6 was reduced to three months, but with tJio 
possibility of rev i c^w .if this time proved insuf f xcien t . 

The submission to OHTJ - *A proposal to In ves t i (^'a te the 
selective dissemination of information (SDI Xnvestigatioji , 
PJiases 3-6), Institution of felectrical En^^ineei^s, December 
1966» is given in Appendix 1A. 

Foliowine: acceptance of the proposal by pSTJ , work be^>;an on 
Phase 3 in July 1^67. However, owiu{; to thc^ noed to recruit 
staff, it was not possible to proceed .fully until the* middle^ 
of S(iptember and, as a result oV this and of pi^objems with 
the comf)Uter s<^i^vlce. Phase d:id not bo^v-i until May 1968. 

Durin(^ the next six months an exptirimeuta I service was pro- 
vided to users. A total of seven computor runs was made ancJ 
the output despatched to participaii ts for comment. 

For various reasons, however, only a small proportion of 
participants rt^ceivcd all seven outputs antJ by the enti of 
six months the system was not functioning;- as W(v| J as had 
been hoped and planned. Faced with tiie choice of f^oJn/v 
ahead with the weekly sc^rvice or delayJn/'; until the coiiiputtM- 
serv.ice had readied a more reliable statc^, it was (h^oided 
that it was of prime* importance to coimiKMice^ tln.^ r(»/viiJni' 
service, even thou/^h i. t iiijglit ho (h* 1\m: t'' vc^ !\-ith(M- that to 
risk disil lus ionmcMit anion/j; tln^ participants. It was, how- 
ever, a/^r(^ed tliat Pfiasc* ^ should not b(^ considoitMl to hav(* 
started until (Mjptain basic rcMpj i in'ukmi ts (^ould Ix^ im^t by th<^ 
compu ter s(^rv i eo . 



Th(»se re(|U lrem«'n ts wt^rc^ in Viirl n(*v<*r mot, and it is Ca i i' to 



say that. Phase* ^5 was lu^vot' eoiiij) teul In any s<msso as 
ori/rh,ally <^iivj.sa/';cd» To iAu^ <^xttMi(, that th<^ s<mvJc<^ 
proviehMl was be Low the i^ixMr.tcM! and pot<uitiaI 1 (*v(; I o 1* 
iMfcc t i vcMJess, the r•<^su I ts o [* tht^ 1 MV<^s t l/va t i can only 
have l)«M'n sci^iou^Iy inipaircMl. 

Siarr 

With tho varied and riuctaatinc demands Liiiposed on a smal,l 
staff in. providin/^ a coiuplox weekly service as well as 
running an I nves titration , the need Tor versatile and hard- 
working people was paramount. It was extremely fortunate 
that t}ie Investigation staff worked as a team and took 
pride in rallying to overcome recurrent crises. Staff 
turnover during the investigation was fortunate jy very low. 

The staff engaged on the Investigation at its conclusion 
in December 19^9 were: 



Manager 

Indoxf^r/Analys t 



P C 3 ague 
I) J Carr 
L Evans 



Investigation 
Secretary 

Floxowri ter 
Operator 

Typist 

Clerk 

C J erk ( part- time ) 

Machine ope ra t or 
(part- time ) 



Mrs A SharLand 
Miss A Thompson 

Mrs P Passenger 

- Mrs S Howard 
Mrs B Piggott 
Miss L Silvorwood 
Mrs K Simpson 

Mrs li Gi^eene 



The Investigation was under tlie general direction of 

Mr T M Aitchison and Mr C W Cleverdon acted as consultant. 
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PREFACE 



Althout^h this report apj)oai\s more* than a year after 
coinplotioii of tho litvos tx/'fation .it lias been produced uiid<\r 
/jroat<»r pressure* an<J to a t.i.(;htor scliedule than one vouJd 
have wished. 

In thf orLf^inaJ proposal to OSTT a j^ei^iod of* sijc nipiiths 
vas aIIoeat<»(i for f*un-.tune work on anaJysis of results, 
addi liotK'iL studies, and wri tin/v and production of tlie 
report. in tlie fvrajii .awarcJcKl uii J 967 this pcMMod was 
rcHJuced to throe inonilis, witli tlu» possibility of further 
tJme b<»i.n/'- inadcr availaf)I.o if r(M}uirc'd. 

In the* event, howovtM-, sliu:e the SDT. Service provided in 
th<i Iuv(»sti/;a t Lou was continued, without a l)reak, as tlie 
pixsent e()st--r<;covej7 service, tlio availability of SDI staff 
for any analysis has been s<wor<*ly limited. In particular 
the writin^^ of tJie report has had to be largely a spare- tiine 
activity carried out undc.^r considerable pressure. 

ApoJoc-ies ai^o necessary and are freely offered for the 
inevitable shorteoinincvs which the iv^port has as a consequence. 

Acknowiedfciuents 

In contx\-ist it is pleasant to be able to acknowiod(;:o the 
assistance /^jiven by tlic- many people who took part in the 
Inves ti/^ution. 
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Invest l/^ation would Jiav(i l)e(ni Impossibles. In paj^ticular, the 
.librarians and ini'ormatlon ofPicc^rs in thoij' or/^anisalions 
who acted as Proj'ccH, Associates destii've our ^v'"a t J. tiK Jc* for 
heipin(i" to coliect tJi" sainpU* of j)ar c i-pan Ls an<J k<M*pin/v ns 
in touch with the n</**(Js and opjnunis of tfuM. r us(m*s. 
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Chapter 2 J 

I 

GENERAL HEVIRV 



In lookin/j back ovoi* the ''nvos t:l£;ation tlio iiiost obvioui^ 
fact is that many changes have occurred in i.ho Tieid oT 
library and infornia tJ.on studios since the fMrst proposals 
for this work were put forward by the National Electronics 
Research Council in Septejnber 196^K It would be surpi"i>sin{;j 
if J in this tijno and in the lif^lit of the rcs(*arch and 
development activ.i ty made possible by OSTJ fundin^i-^ and 
inspired in many cases by such seminal researches as 
those of Cleveruon and liis fellow i-esoarchex-s at 
Cramfield, the context of the SDI Inves ti^fn liioxi had not 
Ciiayiijed consider-^bl y • 

The investi^^ation work formed one of the earliest proposals 
made to OSTI for financial support, at a tijne when selec- 
tive d:? ' semination oV information was a very irecent innov- 
atioi when co-ordinate indexing sclieir<es and thesauri 
w<tro still not In /5:encral use, and when such natural- 
language systems as existed did so in defiance of* accej)ted 
dogma, before the J*evelations of the Cranfield XT studies. 

The large-scale nature of the proposed Inv(».s? tiga t Lon and 
the relatively !axvv<' funding needed, tonded to delay 
the srart of the work. 

Given that the proposai envisaged tlu; I'i.rst two phases of 
the work as prel i jii i jjary manual /'easini'l i ty studJ(!S and 
system - design, it was sensible to fund these scpara t<> Ly 
and to consider f^ujidJ.ng the remainder iAn* w(M-k in the 
iiglit of the x'(»suJts* However such an aj-ran/v^Miien t en- 
tailed hav Lng to d(i (*<* r recrui tmen t o f s ta i'V I'o r the Jiia In 
work until the ini t i n I report could be s tu(i i t-d an(J fund- 
ing agreed. Ilie t.i.mi^ to recruit; suitable .staff foj- an 
investigation can be very long and is of*ttiij the cri tical 
aspect in progranmiJ ng the work . 
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Tn (.!ic» SD.T. ;Tnv<»s L i /»a (; i.oJi , fil thou^^h all t!i<» stnlT on^^a{j:od 
jji Phitst.* 1 ancl ,V exct^pt thcj D.i.rocLor and Ass is tan (: Director 
l(»fM. in March 196^), I.Ijo Jiiain delay In start iiJ^v Phase 3 
was l.h(j ('xtont of* I.!h? financial support to bo provided by 
OSTl and thi! mt?tlit)d oi* iundin^. The matter was Tlnally 
ri»so.lved by transf*oi* of the work to the InstJtut:ion of 
Electrical En^jlnoers to rorm part of j.ts ncwLy set-up 
INSPKC operation and the receipt by the Ins t. L tution o£ 
an OSTl f^rrxnt to support the entire Inves ti/^a tion . 

The delay involved :in working* out these arrangements and 
the need (:o find su L table acconuuoda tlon foi' the Investiga- 
tion at Stevenage, whore the Institution liad ujovcd its 
main publishing and information activities, was not in- 
considerable. Thus it was July 196? beTore a start could 
bo made on Phase 3 • 

The decision by OST.E to abandon the oi-iginal plan or 
contracting all cojjiputer work to a comiuercial computer 
bureau and , ins te'ad , to buy the ri*la tive ).y-un tes ted 
SDI programs Tor running at the new Ly-estab I i slied 
Docujiien tation Proc<:ssing Cen tre was , in I'o t rospec t at 
least, unwise. As a result^major problems arose which 
are discussed a t soJiie Lengtli In Chap ter 8 . 

However most oi' tin- problems mentioned so far are probably 
unique to an investigation conceived at a particular time 
and in particular circums tances and , tJiough genuane to an 
understanding uV tho work, have few general implications. 
Perhaps, there J'ore, it is more rewarding to consider 
other aspects of tlie Investigation. 

Scale of the Investif^a tlon 

The Investigation was from the i'irst envisaged as a lax*ge- 
scale project involving the sotting up and study of a 
regular weekly si^-rv^jce to upwards of 600 ust^ra. Tn retro- 
spect it is possib Le that there was sojiie con flic t in the 
logic of se ttii^g/ up such a large sys tem as the" basis for 
an investigation. One i-eason foi* deciding on the size 
was that a small-scale system could be too (jasily optimised 
to meet the known requirements of its few users aJid would 
in other ways not suffoi^ the constraints of a large-scale 
service. At the sajne tijnt^, however, it was perhaps over- 
looked that in any investigation of an experimental sex^ice 
it is important to be able to modif\ tlie systejn and sucii 
modification is dilTicult when tlie system is Uu-ge. 



liivcs L i.r.n I Loii vorsus Sorvlcu 

Ono or the persistant problems .in the Investigation vas 
hov to preserve a suitable balance botvoon the need to 
maintain the regular service and the aLLocation of staff 
to the vork of inves fcifjatinc that service. This task was 
made particularly di.ITLcult by doficionci.'.s ui tlio computer 
service, and by the lack of an En d ox or/Ana I ys I. durin,; a 
crj.tj.cal part of bUn Liivostiffa Lion. Hov<>vf«r, L<'avin£r asido 
these particular piobloms, it sfMuris cioar Unit In any' 
situation where a service and an investigation ar<> boinc 
run Jointly, the division of effort betveen tli<v two must 
hii properly defined since the demands of tli<> s<-rvice wi L L 
always tend, by their immediacy, to absorb oITorl that 
should be devoted to tlie inves titration. 

It is possible tliut liere again there was an und- rlyjnr 
conflict in tlio purjiose of the .Divos tiga t.i on . On the" 
ono hand the official policy was that tlie Invos tiga t j on 
was a research pr.,Jo(:t in wliLcli tiie studies L.>ok precedence 
over the mere maintenance of the service. At th<' same 
time the results of many studies could only bo useful' if 
the service being used as fclie basis of the studies was 
not vitiated by some fault, including interLupfcioji of 
that service. 

It was also felt tiiat the assessment of the accoptabili t^y 
to users was' at I.e:is{; as Important as :iny other pnrt of the 
Investigation; if if was not acceptable, many of thi? otiier 
studxes became acadtjniic. Efforts to achieve an acceptable 
level of service tlierefore seemed sensible. 

The question of acceptability and the linked question of the 
level of cost tliat people are willing to pav for SDI are of 
course critical to the setting up of a contiiuiing service. 
It was unfortunately not possible to obtain meaningful 
computer timings for the SDI runs and in the light 'of tlie 
INSPEC SDI cost-recovery service programme It was decided 
to leave these questions in abeyance. 

Weekly variation 

One of tlie major tilings learnt during tiie llnve.s I. i ra tJ on 
was the difficulty of carrying out studios on .1 eons tan i. I y 

changing syste I,, an SDI system, tin.- data b;.s<>, <lo<-u- 

iiients retrieved, relevant documents and to some <rjctent 
the search profiles, all change from week to w<iok. S.>m.- 
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studios requiring a real-life situation can, of course, 
only be carried out in such an environment, and for these the 
large nujubor of uncontrolled vai^iables haw to be accepted. 
On the other hand there are many other studios relevant 
to SDI operations vJiich might more conveniently be carried 
out in a different way. 

Co-0[jeration from participants 

Tho assistance giv<Mi by participants in the Investigation 
was ono of the rrrost /VJ'a tif yu n^: aspects of l\\o work. 
JnltLal (Joubts about the feasibLlLty ol' a study based on 
-.regular, long-term co-operation from a large number of 
volunteers were quKckhy dispelled. 



Profile 



Perhaps the most important decision taken in setting up 
the Investigat Lon was to atteuipt to cover tho user«s 
requirements in th(M.r totality. By tho way the statement 
of his information requirements- was invited, and the 
examples given liim, the user was encouraged to seek to cover 
all his main interests. 

This had at least two important consequences. First tlie 
sixe of the profiles was considerably greater than those 
of most other systems, where the ai*" tho profile is 
deliberately restricted. Secor . and undoubtedly the moi^e 
xmportant, th<; i)artxcipant assessed the relevance of tlie 
notifications he received, not to his profile, but to his 
interests. This, ol* course, made it much more difficult 
to provide a satisfactory performance and it is surprising 
that the systems performance recorded on the Investigation 
v/as attained. 



Although there is little doubt that any future SDI services 
are likely, for economic reasons, to limit the user^s 
profile to only oim or two of the main elements of his 
interests, it is considered that tlie decision to seek : 
in the Investigation to 'cover all the user's Interests 
as far as possible, was the correct one^ since it provided 
an upper limit to tho possible influence of SDI on tho 
u.5er. It must be admitted, however, that it iuiposed a 
very great burdr^n on the staff in setting up arui ()j)orating 
the service. 
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SELECTION OF USER AND CONTROL GROUPS 



It was decided that the size of the sample group of users 
of the SDI Service should be 600 of whom 540 were to be 
individuals drawn equally from 

a) universities and colleges 

b) government researph establishments 

c) industrial firms- 

The remaining 60 were to bo small groups of people having 
common information requirements which could be served by 
a single SDI profile. Such groups were typically research 
groups in univers i. tins or In small firms. 

A cori^esponding sample of fiOO people was to be drawn in a 
similar way to act as a control group for soaio of the 
s tudies . 



The work involved in drawi.ng a random sample of 1200 people 
to represent the totaJ population of oLoctronLcs I'os^arch 
workers was considerable. The basic problem %^as r.o discovn 
which people were on/*a/vo(l In oloc Ironies research in each 
organisation and thi.s was not ina(l(,» easier |jy quest iohs of 
security, both industrial and nationaJ. 



Drawi.ng the SampJ e 



Various methods liad to bo adopted fox' drawing the sampJo 



to moot tlie indlvLdual requ Lrouiori ts of dirferont types of 
or/rani sat Ion. Howovor, havin/^ obtained access to a list 
of the relevant people in the oar/van Lsa tions the* procedui^c* 
was as follows: 

1) Number all people on tho lists 

2) Select, usi.n/»; random number tables, an approDriate 
number of names to allow for a required usor\rroup 
and control /^roup with an allowance X^or wasta/;e 

3) Send quest LonnaL re and covorin/v lottoi* (Appendix 
3A and 313 ) to the selected people 

k) Examine replies tOr determines whether tlio respondents 
were working; in the fi.eld of electronics research 

5) Assign valid replies, as received, aJ tornately to 
potential user or control /yroup 

6) Chase non-replies to the questLonnai.ro (Appendix 3C) 

7) Invite potential users to participate In ?;he 
Investigation (Appendix 3*^) 

8) Chase non-rep Lies to tho invitation (Appendix 3E) 

9) Where the number in the potential control /.^roup 
exceeded the number of definite users, select at 
random an appropriate number of people from 
control /rroup and tj*oat as potential users as in 
(7) above 

10) Where size of definite user and control />;roup was 
insuf f icJ en t , select futM;]ier names from oi\i/';inal 
list and repeat the procedure. 

The cumulative losses at the various stages of collecting 
the user and control /vroups were? naturally fairly heavy 
and to arrive at the required total o f approx i iiia tely 180 
users and 180 centre J s involved contacting a much larger 
original number. Thus in tho case of universities and 
colleges the size of the original sample (.0 wljom question- 
naires were sent was 757, of whom 192 became <>v<u)tua! ust^rs. 
Figure 1 shows t}ie figures for each of the three r;roups of 
users. Column 1 gives the number of people who received 
the questionnaires and column 2 tlie number of repli(5s. To 
a large extent tlio suitability of the people mak in/v up the 
sample at this stage r*or use i.n tlje SDI Lnv(\s t i /va t i on was 
unproved since we were opera tin/v on lists ooiiii)ile(I on tlie 
basis largely of other people *s assessment oV what compj-Lsed 



an electronics research worker. With the questionnaire 
returns, however, we were able to see for ourselves the 
type of work being done by people and it became necessary 
to eliminate some as unsuitable. There were, of course, 
various reasons for such unsuitabili ty , but for the most 
part the people rejected were eL<;her not en/5at^ed in any 
type of reser.rch and development; work or were enga^'jed in 
fields which we did not consider to be electronics. Column 
3 shows the number wlio replied to the questionnaire and were 
considered suitable. Of the University people who replied 
to the questionnaire a lar/^e number of those rejected were 
PhD students who expected to finish their theses within 
the next 12 months and were therefore considered unlikely 
to be available for the duration of the Investigation. 

As mentioned earlier, approximately half of tliose replying 
to the questionnaire were Invited to participate in the 
Investigation and receive the SD!' service. Those; \vlshing 
to do so were asked to complete the Statement of Information 
Requirements (Appendix 3F). The numbers invited and those 
accepting are shown in columns k and 5 of Figure 1. Here 
again, with the fuller information given in the Statement 
of information Requirements, It became apparent that the SDI 
Service would not cover the interests of certain people and 
these had to be removed from the final user sample, which 
is shown in column 6. 

It was intended originally to have, at this stage In the 
Investigation, a ratlier larger number of users to allow 
for expected wastage In the intervening period before the 
start of the weekly service. However, as can be seen, the 
extra effort required to obtain more users and, perhaps 
more important, the time required to complete the cycle of 
operations Involved made it impracticable to obtain this 
extra margin of safety. 

Compos i t Ion of the sajitpie 

Undoubtedly a number of the users of whom profiles were 
compiled were on the fringe of the subject area of the 
Investigation. There were several reasons for this: 

a) Difficulty In defixilng precisely what was meant 

by electj^onics research. We had Ilst(ul tJi(^ topics 
we expected to cover and to some extent accepted 
that the user^s assessment whether his int(*j*ests 
lay within the field was, at least initially, as 
valid as our own. 

b) The largo expenditure of time and effort required 
to select users and obtain information requirements 
s tatements, coupled with the urgent need to complete 
Initial profile compilation and tes tiiwv, teiided 
towards a widening of our definitions to allow 
inclusion of users rather than their exclusion. 
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To soniP oxtorifc this mod LfLca t i.on in definition of the 
subject coverage was l.o be expected and the selection of 
articles oould easily be adjusl-ed Lo include the now 
topics. However, i.n some cases these fringe interests 
could only have been adequately covered by a major expansion 
of covera,ve. This was not possible and the coverage of 
these .interests was bound to be partial. 

SamplJmr methods for universities and colle/^es 

Since (.here was no complete and up-to-date list of people 
engaged on electronics research i.n British universities it 
was decided to obtain this Information direct from the 
rolevani, depar tmcnt.a . 

A letter (Appendix y.) was sent to the heads ol' departments 
.in universit.es and colle/yos where it appeared li.kely that 

ittl^ZTr^ ^"r^^^^^^' ^^^''■"'i- Appropriate departments 

included Electronics, Klectrical Engineering, Physics, 
Electron Physics, En,.ineering etc. Each head of department 
was asked to provide a list of all people engaged in 
electronics research (as defined in the letter) including 
lecturing and research staff and post-graduate students. 
Only those people expected to be still in the department 
by December I968 ( tlio expected terminal date of nie 
•Investigation) w(>re to be included. One-hundred-and- thirty 
exght letters were srnt and 125 replies were received from 
/« separate institutions listing a total of 1 362 research 
workers. The in.s 1 1 tu tions and the number of people listed 
by each department is shown in Appendix 31I. 

From the 1 362 people it was deci.dod to select k^O names to 
give the required .180 users p.lus 180 controls and an extra 
JO to allow for wastaj-e. Some 71A of the total were 
concentrated in 1 'J universities and colleges which had more 
tnan 30 researcli workers each. From these lar/.er 
xnstitutions it was decided to select both usei^s and controls 
but Irom those havm/v fewer than 30 research workers it was 
deci.ded to select e i.-( her users or controls, but not both. 
Ihe 13 institutions made up just over half of the total and 
the same proportion was preserved in selecting; the sampJe 
'■""'oon i-'-'^'^ 'o select 23O from .the 1 3 and the remain- 

ing 220 from the other 62. Since the I3 listed a total of 

research workers the 23O required were chosen by raiidon.iy 
selecting a tliird of those in eacli. 

The rema.ining 62 inslitutlons were numbered and selections 
made at random until the total numl,er of researcli workers 
m those selected reached 220. This total was made up from 
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20 ins (; i. (.lit Ions* A i>ocoiid select i.oii was tlioii inado from 
among this 20 to do t<* rini.no whLcJi sJiould prov i do usors and 
which the control ^roup. 

Apart from simpliTyln^r the vLsltlnc* of users by the SDI 
staff, the conceiifcrat ion of usors or controls in separate 
institutions was chosen to avoJd tlio affect of the transfer 
of information botwoon the users and controls and to allow 

comparison to be made witli results in the 13 institutions 

which pi^ovided both usei-s and controls • 

In the light of the response to ilie questionnaire 
(approximately 6() percent includln/v 10 percent Trom research 
students who wor(^ unsuitable ow i.ng Co the fact that tliey would 
not be available up lo the end of the Investigation) it was 
decided that a furtlior sample would be neededi 

Accordingly a furtJicr 75 were selected from the I3 un Ivei^si t les 
having more than 30 research workex^s. A Curtlxer 78 were 
selected from another- ei.ght universities and colleges, three 
of which were chosen to provide users. 

By mid-Noirember 1967, i'rom the total of 602 quos tiomiaires 
sent, 160 sultab.ie usors had been obtained and 162 members 
of the control group. Althougli a small number of user 
statements were stil.I arriving it vas thought unlikely that 
the required total oV 180 in each group would be reached. 
A further random sample was therefore taken from those 33 
universities and col.Ioges which had not been previously 
selected to provide participants. Selections were made 
from these at rajidom until those selected made up a total 
of 145 research workers. A second sample was then taken 
of the 11 contrii)ut lag organisations to dec:t.de which should 
provide users and which controls. By this means names of a 
further 76 potential users and 73 potential controls were 
obtained. 

A suimiiary of the universities pi-oviding users and controls 
is given in Appendix 31 showing the numbers invited and the 
total replies to the questionnaire and to the invitation to 
take part • 

Sampli ng methods for government establishments 

For government establishments a somewhat diflerent method 
had to be adopted. A list of suitable organisations thought 
likely to 1)e engaged In some electronics research was drawn 
up and a letter sent to the director of eacli to seek approval 
for the participation of personnel of tlie organisation to 
take part in the SDI investigation. A list of the organisa- 
tions approached Is sliown in Appendix 3I. 
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In nearly all cases the or^janLsafc i.ons had a librarian or 
information officer who had expressed villin^^ness to act 
as Project Associate provided that his Director's approval 
could be obtained. The assistance given by Project Associates 
was important throu^jhout the I.nves tigation but was particularly 
vital during the crucial stages of collecting the sample of 
participants. 

It will be realised that, with juany oV the organisations, 
considera*t lo:as oV security made It impossible to obtain a 
list of the people cn{';a,(^Qd In electronics researcli frojii 
which to draw the sample . In these cases , thoi^ef ore , the 
ac tual random sample liad to be made wi thin the organisation 
by the Project Associate and tlic» names of selected people 
sent to us to contact further. Tristructions (Appendix 3l) 
were sent to the Project Associate to guide li Im in selecting 
the sample. 

The actual number of pai^t Ic Ipants to be selected from each 
organisation was chosen by SDT staff on tlio basis of an 
estimate of the likely amount of electronics research being- 
carried out there. These figures had to be adjusted as It 
became apparent that sojue organisations would be unable to 
supply tlie estijiiated number of suitable candidates. The 
final sample obtained therefore cannot be saJ.d to represent 
the exact proportion of electronics reseai^ch workers In 
each of the govei^nnient establishments through a number 
contributed by each would generally be in relation to the 
amount of electronics research carried out. 

Details of the part ?^:C Ipat ing organisations and the number 
of user's and coxitroli> contributed by each are given in 
Appendix 3I» 

Sampling methods for Industrial firjus 

In selecting the sajnple of users and controls j'rom 
industrial firms the general problejn, as with universities 
and government establishments, was ihat there was no simple 
way of establishing how Juany electronics i^esoarch workei^s 
were employed by each firm. However, it was assumed, for 
lack of better iuf orjiia t Ion that the amount of research and, 
therefore, the nujuber of research workers would be in 
proportion to the size of firm, i.e. nujuber of employees. 

This latter inforjiia t i on was available for tlio olcctrojiics 
industry in the Kuroloc Pocket Ouide I965. Mollis divided 
firms- into four /v^oups as follows: 

Group 1 firjiis with over 10,000 ejuployees 

Group 2 firjus with between 3»000 and 10,000 employees 
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Group 3 firms wit,h between 1,000 and 3,000 employees 
Oroup A ri.rifis wUh Loss than 1,000 einpaoyecs 
Group 1 ririns 

The ten fLrins in this /vi^oup employed to^jethor 62. h. perceUt 
of the total employees of the el<!ctronics indiislj-y. 11 was 
decided therefore, to select from tlieso firms 23O {'62. h 
percent) of the 36O users and controls required from 
Industrial firms. These were to l)e drawn cqnaJ ly from each 
of the ten firms, i.e. 23 from each. To allow for wasta^^e 
it was decided to attempt to obtalti initially 30 from each. 

The firms In Group 1 were : 

Associated Electrical fndustrles Ltd 

En^jlish Electric Co Ltd 

Elliott Automation Ltd 

Ferranti Ltd 

General Electric Co Ltd 

International Computers and Tabulators Ltd 
NV Philips Gloe i lamponfabr iken 
The Plessey Co Ltd 
Pye of Cambx^ld^^e Ltd 

Group 2 

Group 2 firms employed 15.7 percent of the total employees 
of the electronics industry. Fifty-six (15.7 percent) of 
the total users and controls were, therefore, to be selected 
from firms in this ^^roup. However, to avoid dealinfv with 
too many individual firms, only six of the firms in this 
group were selected at random. Prom each of the six selected 
firms It was decided, therefore, to select 10 people, wliicli 
number was made up to I3 to allow for wastage. 

The initial firms selected were EMI' Electronics Ltd, 
George Kent Ltd, Honeywell Controls Ltd, Spongy Gyroscope 
Co Ltd, S. Smith & Sons Ltd, and Associated Semiconductor 
Manufacturers Ltd. Two of these, Honeywell and Smith, 
were unwilling to participate in the Xnves ti/va t i on and a 
further sample produced Tlxorn-AEF Radio Valves and Tubes 
Ltd and Rank Organisation Ltd. 

Group 3 

The Eurolec Pocket Guide listed 25 firms in f^roiip 3. As 
these, together,, employed 10.6 percent of tlio total 
employees In the electronics industry, hO (IO.6 percent) 
of the users and controls were to be selected from tliem. 
Six firms were selected at random but two of these, 
Foxboro Yoxall Ltd and Teddington Aircraft Controls 
"were considered to be, strictly speak In^v, not. e.l <»e tron Ics 
firms and a further selection was made. Th(^ fi.naJ six were : 
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mi/^or and Watts Ltd 
J^cd irrns ion I^osourch Ltd 
UJ. tra Klect Iconics Ltd 
Hacal Electronics Ltd 
A C Cossor Ltd 
Texas Instruments Ltd 

Si.x people were to be selected from each plus auotlier two 
to allow for wasta^^c. 

Group k 

Approximately 90^) Tirins were Included In (Jrouj) 'l , rarvyln^r 
from one-man bus inessos to .lar;.;e or^anlsat, i ony sucii as tljo 
Guided Woapons Division of the British AircJ^aPt Coi^pora t Ion . 
Al toge tlier the Group h U Irms were I Is toci as cMiiploy i nr; ^ • 3 
percent of tlio total electronics Industry oiii]) j oyees» I was 
decided In general to draw citJjor users *or controls Px^om oacli 
firm, but not both as liad been done wltli IMrms In Groups 1 , 
2, and 3» Because oP tlie very lar^je number oP Plrins a 
random sample was made to produce a very short list of hO» 
A number oP these had to ])e rejected either because they were 
too small to do any research or because tljiey wore subsidiaries 
of lar^^er firms In Groups 1, 2, or 3* AftoJ^ this screening; 
process some 26 firms remained and it was decided ,to invite 
these to supply names of electronics research workers and to 
select the ^0 users and controls from tlie names supplied by 
the firms who replied. 

Some px'oblems occurred In obtaining lists of olectronlcs 
research workers Prom Industrial Piriiis. Tlioso problems 
arose from two soux^ces. 

a) unw 111 In^vness of Pirms to coimnunlcato suc]i 
coimiiercially valuable information to an 

ou ts Ide oxv^an i sat Ion 

b) difficulty in supply In^j;: tills information since 
it was not available In precisely the required 
form 

Again the librarians and information officei's were extremely 
helpful in solving this problem and were, in inos t cases, 
able either to supply a suitable list or to /v^l^ access 
to one from which they could make the necessary random 
^.election under our Instructions. The ins ti'uc t ions are shown 
in Appendix 3L» 

The organisations approached and tlie numbers oP people 
selected from each for the user and control /yroups are 
shown In Appendix 3^. 
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l-Oc(;ioii of parftic i.r>.-iiil..s i.n l.hc SDT Invos t i I. i. 







QUESTIONNAIRES 




IK-VTI'ATTONS 


Type of 
Or/.^anisfi tion 


Sent 


i(e tu J iM*d 


Sii i ImIiIo 


Son u 


tU' Ml mod 

iil 

■.H)G (75) 




Universities 


737 


6i6*(8i ) 


5 1 5 


27 5 


192 


Government 


812 


58 {i:>) 


519 


283 


21.5 (70) 


196 


Indus try 


819 


502 (6i) 






!''<; (82) 


191 


Totnl 


2:i88 


1703 (7i) 


li»8l 


802 


620 (77) 


579 


^ Includes PhD 
].o.''Vo beToro 


sLudont-S \^^]^a lu'id lo be excluded siiict* ij,^.^ u.-j-o J.i 
Ciie end oV t)ie Ijives tt/^n lion 


k (J ,1 y to 



Chap tor h 

LOSS OF PARTICIPANTS THROUGHOUT INVESTIGATION 



One of the most serious considerations in pJ annin^j the 
Invosti^jation was tlic possible loss of participants during 
the course of th(» Invcs ti^^ation. It was of considerable 
importance that the niany aspects of the work which depended 
on the availabiJ Lty of data from a larf^e-scale sample should 
not bo adversely affected by loss of participants. Advice 
on the likely wastajjc rate was souc;ht from various sources 
though tliero appoai^ed to be little directly relevant 
experience. On the basis of known questionnaire return 
rates the possibility of havint: more than fifty per cent of 
users left at th<» end of the Investiffation appeared slight. 
However, it was simply not practicable to allow this size 
of margin in the sami)le of participants. After consideration 
of the factors involved an estimate of thirty per cent wastage 
qver the period of the Investigation was arrived at. The 
reasoning behind this figure is sot out in Appendix ^A. 

In fact, of the totaJ of 571 users for whom profiles were 
originally compiled ])otween July 196? and March I968 some 
^473, i.e. 83 per cetit were still actively taking part in April 
1969. (Actively meant that they had regularJy been returning 
weekly relevance assessments and were not more than four 
weeks In arrears). Using a less strict definition the total 
number of current usors was approximately 500 ♦ Figures for 
each participating crgcxnisation are given in Figure 1. 

The definite known losses can be broken down as follows: 

Left tlie organisation, or changed job, 2^ 
died, went abroad etc 

Deleted owing to nou-re turns ^7 

Dropped out for no known reason ^ 

Withdrew cvi/iig to pressure of work or 

dissatisf ac tion • * 



Withdrew at our suggestion - because 

of fringe interests 3 
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Chap tor h 

LOSS OF PARTICIPANTS THROUGHOUT INVESTIGATION 



One of the most sorious considorations in pJ annint^ the 
Invos ti(^ation was tlio possible loss o£ participant's during 
the course of tlio Invosti(^ation. It was of considerable 
importance that the many aspects of the work which depended 
on the availabiJity of data from a Jar^^e-scalo sample shouJd 
not be adversely affected by loss of participants. Advice 
on the likely wasta^je rate was sou/jht from various sourco^i 
though there appoar^ed to be little dix^ectly relevant 
experience. On the ])asis of known questionnaire return 
rates the possibility of having more than fifty per cent of 
usei^s left at the* end of the Investi/jation appeared slight. 
However, it was simply not practicable to allow this size 
of 'margin in the sample of participants. After consideration 
of the factors involved an estimate of thirtj'' per cent wastage 
over the period of the Investigation was arrived at. The 
reasoning behind this figure is set out in Appendix ^A. 

In fact, of the totaJ of 571 users for whom profiles were 
originally compiled between July 19^7 and March 1968 some 
473, i.e. 83 per cent wore still actively taking part in April 
1969. (Actively meant that tliey had regularly been retu-rning 
weekly relevance assessments and were not more than four 
weeks in ai^roars). Using a less strict definition the total 
number of curt'cnt usots was approxJ.mately 500. Figures for 
each participa cing organisation are given in Figure 1. 

The definite known losses can be bx^oken down as follows: 

Left tlie organisation, or changed job, 22 
died, went abroad o**.c 

Do l.e ted owing to non-re turns 1 7 

Dropped out for no knovn reason 5 
Withdrew owing to pressure of work or 



dissatisf ac tion 



Withdrew at our suggestion - because 

of fringe interests 'J 
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Tho wasta^jo rate was Least for universities and collegois 
and greatest for industrial firms though the difference 
was slight 

Universities 169 from 195 {^7%) 

Government ostab L Ishments 156 from I86 (8'i^) 

Industrial firms 155 from 19O (82?^) 

Strangely the closing of the AEI Research Laboratories at 
Rugby where we had none original users did not seriously 
contribute to the greater loss in industrial users since 
after their dispersion to various locations, six of them 
continued to take part • 

It should be remembered that the university participants 
were first asked to participate early in 196? and a very- 
large number first submitted their statements of Information 
Requirements in May/Jiine of that year* Since various 
difficulties (chiefly associated with the computer) delayed 
the start of the experimenLtal se.rvi.co (piiase 'j) the Tirst 
notifications were not sent out until April/May 1968. 

During this period very occasional contact was maintained 
by asking users to mark lists of dociiments against which 
test runs of their drcxft profiles could be evaluated. This 
lack of regular contact meant that users dropped out during 
this period and the extent of this loss was only discovered 
when users failed to reply to tjie experimental service 
notifications . 

Regular cojninunication would have overcome this problem but 
given tho circumstemcos such coiimiunications could have been 
only of an apologetic or temporising nature and it was felt 
wiser to restrict communication until the situation was 
more hopeful. 

Hie figures for profiles starting experimental service 
includes all users whose profiles had been tested and 
modified ready for service and who were not known to have 
dropped out for any reason • 



The abbreviations used in Figure 1 are: 



QMC 


- 


Queen Mary College 


AUVE 


- 


Admiralty Underwater Weapons Establishment 


BBC 


- 


British Broadcasting Corporation 


CEGD 


- 


Central Kleatrical Generating Board 


CfPO 


- 


General Post Orfice 


NPL 


- 


National Physical Laboratoi-y 


RAE 


- 


Royal Aircraft Establishment 


RRE 


- 


Royal Radar Establishment 


RSRS 


- 


Radio and Space Research Station 


SERL 


- 


Services Electronics Research Laboratories 


SRDE 


— 


Signals Research and Development Establishment 


AEI 


- 


Associated Electrical Industries 


BAG 


- 


British Aircraft Corporation 


GEO 




General Electric Company 


ICT 




International Computers and Tabulators 


STL 




Standard Telecommunications Labora-toris 


TMC 




Telephone Manufacturing Company 



From Figure 1 it can bo seen that loss of users was 

^^^"^ AEI Rugby, GEO Wembley and 

ICT Gorton. Apart from AEI where as mentioned earlier the 
Closure of the Research Laboratories at Rugby caused problems 
xn keeping contact with participants, the high wastage of 
S^^;'-'''-??"*^ appears to have been associated with postal 
difficulties presumably within the organisation itself.' 
When the failure of coiiununication was discovered a number 

of- "^u^Tf- ''^^""^ "^""^ reinstated, notably in the case 

oi GEC, Wembley. 
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Note: Pi.{;ures relate to place at v/liich people wore rocrtiLtod 
even thou^^h they moved later* 
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Note; F'iguros relate to place at wJiich people were rocruited 
even though they moved later. 
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Note: Figures relate to place at wliicli people wore recrui tod 
even though thuy moved .later. 
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Figure 1 (contd ) 

Or (r^ui But to ns parCicina ll nr in the: Tnyns ti^.r. t..- ..^d loss 
of pnrt Icipfjnt s ^os s 
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Note: Fi^jures relate to place k\t \.hjcli people w(sre rocruitcd 
even thouf:h they moved Ic'jtcr. 

^-8 



Chapter 3 

l^EASONS FOK LOSS OF l^AHTJ.CIPJVNTS RECIDIVING SD'i: SERVICE 
NOVEMIUCH 1968-NOVEM13ER 1969 



Althout^h Iho ral;<^ oV Loss of participants thx^ou{jhou i: the 
lnvesfci{.^ution fel.i within the expected limits it was 
decided to look more* closely at the reasons for this loss 
and in particular th(j loss duriji^j the period that users 
were receiving rci^^uJar weekly sei^vlce, November 1968- 
November 1969 • 

In the earlier phasc^s of the Investigation, which were 
devoted to collecting the sample of users, obtaining 
information requirements, compiling and testing profiles, 
there was little immediate advantage to any participant 
in continuing to give; up his time to the Investigation. 
However, from the time the weekly service began to be 
provided it was obviously of considerable interest to know why 
people receiving such a service f ree-of-charge should choose 
to drop out, 

A study was therefore made of those 69 participants who 
withdrew from the Investigation between November^ 1968 and 
Decembex' I969 to discover their reasons for doing so. 

Method 

The records. of Che 69 participants wore examined and 
compared with those of the total sample whore appropriate 
to discover the reasons wlie ther stated or underlying which 
led to their wi thdrav/al from the Investigation • 

On examination it b<^came clear that eight oV the 69 coul.d 
really be considere<J not to havo par tJ cipa tcul i n the 
weekly service . Al though no ti (\i.CiX t i ons wer(» son t to thorn 
for a sliort tim<j thoy had not at this time oj* In tlio 
previous experimented 1. running i)erio(J returned any rcHcvancc 
assessments. As a result their profiles had nocessar^ily 
remained untested. Their withdrawal could not, tJieroforo, 
be considered' to provide evidence of the accoptability 
or otherwise of the service . 
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Uh- rcMiialriin^^ 6l par L Lcipan ts wore round to fall into 
six caioc:orics according to the ostensible reasons for 
th'*jr withdrawal* 

1 ) The participant's main field of interest fell outside 
the coverage of the SDI Service 

Jji most cases these ])cople had been included because part 
of* their requirements were within our expected covera^je 
cind the pressure to ni-ike up the sample of participants 
dictcitcd their inclusion rather than exclusion. 

Five people fell Into this catot^ory, four of whom withdrew 
oil ctdvLco from the SDt staff that the poor service was 
unlikely to improve owing to the nature of their interests 
and the coverage of the service. 

2 ) Participant left the United Kingdom 

ISlovon participants dropped out as a result of leaving the 
coimtiy to work abroad. 

3) Participant chanr^od the nature or place of employment 



Nineteen users foil within thi-^s category, eleven of whom 
withdiow as the result of leaving their place of employnient . 
Six more foiond that their information . requirements or 
duties had changed so as to make the SDI service inappropriate, 
or the remaining two, one was beginning a sabbatical year and 
for the other no reason could be discovered. 

) Pressure of work 

Si^ users withdrew because pressure of work made it difficult 
for them to continue participation. 

5 ) Satisfactory service already beinj; received^f rorn other 
source 

Tt was to be expected, particularly with those participants 
from government establishments and industrial firms, that 
many would be receiv:! ng an information service within the 
organisation which would overlap to a greater or lesser 
extent with the SDI Service. 

Five users withdrew because they X'elt their interna.l scx^vicc 
was better. In view of their subject interests it could bo 
expected that three of these would not recei.vc very compre- 
hensive service lYom the SDI Service. One other was 
dissatisfied with thc^ service coverage in that research 
reports were not included. 
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6 ) Part icipants TaJ I cd to retux^n no fcif icalloiis rcfvulariy 



Participants were expected to provide regular relevance 
returns to allow ass(*ssment of the service. Those who 
failed to do so for a given length of time and after 
suitable reminders W(*re dropped from the Investigation. 
Fifteen of the 6l were dropped for this reason. Of these 
seven were university participants whose returns may have 
been affected by vacations. 

Underlvin/c reasons 

Although the reasons given for withdrawal did not appear ij 
point to any dissatisfaction with the service it was 
considered that the ostensible reasons might be only part 
of the story. It was possible, for example, that the excuse 
of pressure of work, meant in fact that the time required 
to participate was not thought wox*thwhile in relation to the 
value of the service* An examination was, therefore, made of 
the background to each profile to see whether evidence of 
user dissatisfaction might exist. Sources examined for such 
evidence were the conunents of the user during the service and 
at the time of withdrawal, performance figures for the 
profile, user rating of the service as expressed in the 
questionnaire, and the user's field of interest. The relevant 
information is summarized for the 69 participants in Appendix 
5A, divided according to the reasons given for withdrawal. 

Group 1 

In the case of those main interests were outside the scope 
of the service four did not in fact withdraw voluntarily, 
but were advised to do so by the SDI staff. In the case of 
the fif^th. his requirements were for specific engineering 
design data which it was not the function of the service 
to provide. 

Group 2 

Leaving the country seems an adequate reason for dropping 
out of the Investigation and no underlying reason needs to 
be sought. Some did express a wish to continue receiving 
the service abroad, but failure by the others to make this 
comment cannot reasonably be construed as a.n unfavourable 
reflection on the service. 

Groups 3 and k 

In the case of those who gave changed duties, changed 
subject field or prc^ssure of work as the reason for with- 
drawal it is not possible to be sure that a poor opinion 
of the service did not contribute to their withdrawal. 
Even in the case of those who changed their place of worlc 
this in itself could not be considered a complete reason 
for withdrawal since* the opportunity to continue partici- 
pation was offered where we knew beforehand of the intended 
departure . 
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Those who withdrew because they were receiving a more 
satisfactory service elsewhere were most definitely passing 
adverse comment on the SDI Service. Nevertheless, given 
their subject interests they were undoubtedly correct in 
feeling that the SDI Service would not provide coverage 
comparable to those other services • The lesson to be 
drawn is that the initial sample could well have excluded 
them since their subjo*ct interests lay outside the scope 
pro'por of the SDI Service. 

Group 6 

Thou{^li failure to roLurn relevance assessments does not in 
itself indicate dissatisfaction with the service it does, 
(jiven tho concomitant throat of losing the service, indicate 
that the service is not enjoying the participant's highest 
regard. In addition the lack of feedback resulting from 
non-reti;rn of relevance assessments would, where service 
was unsatisfactory, serve to perpetuate this situation. 
In some cases the lack of returns was duo to poor postal 
arrangements . 

Of the fifteen people in this Group, seven had earlier 
returned questionnaires in which like all other participants 
they wore asked to indicate that aspect of the service 
they found least satisfactory. On examination it was felt 
that the points they mentioned were valid criticism of 
the service they had received and could have contributed to 
thoir withdrawal. 

The available inforjnation relating to each of the people 
who withdrew is summarised in Appendix 5A. This includes 
the type of organisation (university, government or 
indus^;ry), the date the service was suspended, the partici- 
pant's rating oV tho service in his questionnaire, any 
relevant conunents and the percentage Precisxon (P) or 
Recall (r) performancci figures as available and the series 
of weeks to which they refer. 

Correlation of user satisfaction with withdrawal 

Xr it is assumed that dissatisfaction could be an underlying 
reason for withdrawal it is useful to examine the assessment 
of tho service by people who later withdrew. . 
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The questionnaire assessments are shown below for each group: 



Group 


VG 
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P 


P 


VP 


Not 
Given 


Total 
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i 
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5 
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1 
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5 
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15 


Total 


3 


13 


18 


5 


2 


20 


61 



Fi/^. !• Assessment of service by sample of 6l participants 
who later withdrew I'rom the investigation. 

The rating of the sex^vice by those who withdrew is shown in 
Fi{;ure 3> compared with that for all ^23 participants who 
completed this part of the questionnaire. 

Taking the groups where the precise reasons for withdrawal 
are not unambiguously clear, i.e. Groups 3i ^ and 6 the 
figures are: 

Pig. 2. Assessment of service by 21 participants who later 
withdrew from the Investigation. 
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.1 


21 


Percent ago 


9.0 


2if 


62 


5.0 
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These figures show onJy 33 cent of the ^1 rating the 

service Good or bettor compared with almost 53 per ce.ot 
for the total sample of ^23« On the other hand 62 per cent 
rated it Fair compared with 3^ per cent. The question, 
therefore, largely revolves round the meaxiing of "Fair". 
However, it can hardly be taken as an enthusiastic comment, 
and oven allowing for the very small sample there seems 
to bo some reason to suppose that; a somewhat poorer service 
may have boon oxpcri(Miced by these I participants compared 
with tlvii sample as a whole and this may have contributed 
to their withdrawal* 
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Chapt'er 6 



SDL System Operation 



The SDI operation was desl(;ned (.o provide a v/oekly service « 
to users. The operation revolved round a fixed weekly 
computer run at which all file updating^ as well as the 
matching; of the current documents a^yalnst the profiles was 
done. The data preparation was done by the F.NSPEC SDI 
team at Stevona^^o and computer facilities at various 
locations were used durlnf^ the course of the Investigation, 
but all in or near Manchester. 

Aim of Iho operation 

In or/';anisln;; lha op<-ration there were two cliief aims: 

1) To iiilniiiilze the delay between publication of 
the documents and tholr notification to users 

2) To update users' profiles as c(uickly as possible 
whc.Miever modification was x^equLred 

A number of factors In particular served as obstacles to 
these alms: 

a) Limited punchln/v capac^lty of the sinr;le Plexowrltei* 

b) Sinr;le weekly computer run 

c) Delay In transport of material to and from the 
computer at Manchester 

Document Supply 

As mentioned els<*w]ier(; th<! docu!!!t5:it input to Uiv .syst(iiii 
consisted of Enr;l i sh- lanr;uar;e p<?rio(lical artin^(»s. Copies 
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oV the r>c-ri i cfi I s r-(M|uirud f'or scajinln^; wore obtained 
I^i-ojii Mi(» Acquisitions Suction of ScLonco Abstracts which was 
sLtuatdd in S tov(Mia^;;c» i.n a separate buildln/^ some five 
minutes walk away. In view cf the importajice of a x^apLd 
service for SDL the relevant periodicals were jnade avail- 
able to SDI immediately after they had been roceived and 
recorded by the Acquisitions Section. However, to avoid 
any danger of losing these periodicals, the originals were 
sent only whore two copies were available. In other cases 
th(.» contents list was photocopied and sent to SDL for selec- 
tion of the articles required. On return of the jiiarked 
contents Jist tlu; relevant articles were copied and sent: 
to SDI. This naturally Introduced delay into tlie system 
at th<j start sinc(j in most cases approx iuiately 36 hours 
would be needed to compie te th is pax^t icular cycle . Where 
a second copy of the original was available tlie delay was, 
of course, much less. However, In all cases a photocopy or 
the soJected article was made and this becauie the uiaterlal 
on which the Indexers worked. The photocopies were required 
In the I Jivesti gat Ion for later analysis after the original 
copies Iiad been returjied to Science Abstracts. To reduce 
the photocopying, only the first three and last two pages 
of each article were copied. in many cases this Included 
the full article but even for longer articles It gave 
sufficient inf ormat i on. f or the Indexers to work on^ 

Sslec t ion 

In most cases the selection of material was carried out by 
the Manager of tJie investigation or, in his absence, .by the 
senior Lndexer/Analys t. From tlie periodicals (listed in 
Appendix 6a ) the articles selected had to meet the following 
cri tor la : 

1) Within the field of electronics research as defined 
for the Invest Ir.at Ion 

2) Yn the English lan^juage 

3) Of at least one page in length and normally by a 
stated author. The length requirement was relaxed 
In respect of certain high quality periodicals and 
In particular the "quick announcement" type such as 
Physics Letters wjiere the articles tend to be very 
brief. 

in general, articles devoted entirely to describing a 
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particular product oV a manufacturer were rejected. 
Docunicjnt data preparation 

Aftor Iho docuiiUMits wore photocopied a docuniont Input slu»ot 
vas attaclicd to oach on which was typed tho title, author 
and c i (at ion details. In the la tier the standard ScLenco 
Abstracts journal abbrev Lei t ions were used. The: documents 
wore then passed for Lndo:cLn(5 and from this sta/^o each 
could then be dealt with as an independent item, i.e. there 
wa.s no attempt to keep together articles from the same 
period LcaJ . 

With tlie index terms assi^'iied by the indexoi s, the document 
in^iut sheets were numbered serially and passed to the tape 
typist. The information on the input sheet was punched and 
any errors that the typist could see from the hard copy were 
corrected immediately. The hard copy was then checked and 
errors sent back for correction at a suitable time. 

Before despatch to the computer centre the individual 
document tapes had to bo re-run throu^^h the Flexowrlter to 
form lon{^ combined tapes containing' approximately 70 
individual document details. This was a requirement for 
ease of handlin^^ by the computer operators. At the same 
time a hard-copy listing,*; was produced in two copies, one 
for despatch to the Documentation Processing* Centre with 
the tapes and one for our own records. 

The need to produce the combined tapes and the hard-copy 
listing' was unfortunate in that II absorbed Floxowrltor 
time tliat could ill bo spared. However, with the over- 
present danger of run failure, provision of the listing gave 
DPC the necessary data for immediate failure analysis, 
thus minimizing the (ioiay. 

On the morning of t)ie day for despatch of material t.o the 
computer, the last of the document tapes were combined 
and the document tap(?s sent off tog;ether with the i.apes 
for updating the pro Piles and tJie descr ip tor V lie • 

The tapes were despatched on Friday afternoon, thus /^Ivln^v 
two cloar days (Saturday and Sunday) for thc:m io ariM.vc 
at Maxiches ter in t i mo f oi* the scheduled compu tor tux\ a t 
lunchtime on Monday. However, it proved Lmpo.ssi.ble to roly 
on this and the scheduled run Jiad to be postponc^d till 
Tuesday lunchtimo, thus incroasin/v the delay by ono day. 
Tn the best circumstances the output l*i*om tJie vuu would be 
received at SLev(;na/';<* on Thursday just bef'oro mid-day. 
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Th() /rcMKjiiil lonii of ont.piil l^roiii Iho computer Listed the 
luiiiilx'r.s of (Ji(^ (iocnimont.s him toll i rjft- a /rjvon profiJo. The 
JM^tunl (h^tniKs of tluise docimionis were provided in the 
f^onii of carols whlcli were ])roduced at Stevenage. 

Tho ori^rinaJ. paper tapes for the individual documents remained 
at Stovena/^(^ afM;or the reproduced combined tapes had been 
despatcliod to Mancljes tor • IJurin/^ the poi-iod from despatch 
op th(^ tapes to receipt o!" ( In^ computer* output the individual 
tapes wore WhI llirou^vii tlio Ploxowritor to print the document 
details foiM*-up on to Mu I t i I th masters* The imistors wore 
tlion run o fM* on two-slioot wlii to board L?" x 8" in size, whic!) 
wore then power-/r,nj ) <, LikmI to produro the 6" x ^1" no t i cfi t i on 
cards. 



The cards were ])laee(l in numerical order into pigeon lioLos 
in an array 15 liorizontaJ by ^^0 vortical designed to allow 
a seated clerJ< to select from any hole. 

Mailing and despatch 

The postal ar ivingemen ts rtMjuirod for an Investigation involving 
a weekly service to eaoii of 600 participants are in no way 
trtvial. Among otiior (hin/vs they entailed ensuring a weekly 
supply or 600 addrtissnd envelopes for despatch of ^notif ica t.i ons . 
Those were providcMl by JKIv headquarters at Savoy Place on their 
addressing macliino using address stenrits cut and kept up-to-date 
by SDl staff. Since llie a<ldressing work formed only pai^t of the 
operation of the headcpia r 1 cm's department concerned ^ the regular 
and urgent supply of so many atJdressed envelopes was not always 
easily compafablo with thc^ir other work and occasionally this 
aspect of the sej^vlce caus<ui problems. 

On receipt of the continuous eomputer print out the three copies 
of the notifical ioiis were decollated and gui.l lo t ined . Two copies 
of the notification were ma I ed wi tli the appropriate user's 
envelope and tin? ihi.rd ropy IMJ ed as a record of the notifications 
sent to the us(m^ for the particular week. Using the list of 
document numbers printed for each user the appi^opriate 
notification card,'^ were selected from the card sorting racks 
and placed together witir the two copies of the print out, tlie 
addressed envelope and a rr^ply-paid envelope ready foi^ recording 
and despatch. 

Data recordecl . 

For eacli user a r(HM)rd was kept sbowin/r wln^tliei- he hacJ h(u>n sent 
notifications for ;iny /viv<Mi week. Whertj no matching; ha<l been 
produced for any \)roV\\Q Uiis fact was also reoordcMl. 

On t.he same slnuit a r(MMird was made of tlnj reievancM^ returns 
received for oacb user and oadi week, showing the PrcMM.sion 
pei^formance as K(^levan< e I and Relovanee ;? documents exj)i*esse(l 
as a fraction of Ihe total notifications sent. These records 
ai lowed the service receivod by each user to be ass(>ssed 



quickly and action to be taken where necessary to improve 
performance or an examination to be made where few notifications 
were being produced. 

In tliis connection it was discovered that the Precision 
pcrlof-mance fi^^urc bn.sed on R1 and R2 documents combined could 
hido cnsos oV very poor HI Precision. From week 030 onwards 
therefore Rl Precision figures were also recorded separately. 

For each week a copy of oarl? document input was kept together 
with its indexing sheet to allow subsequent analysis. Using 
the notif ica t i.on cards a file Ln serial number order was kept of 
all documents input to each weok'r. run and a file by periodical 
title and Jssuc nujuber showing whicli articles had beeji sclectod 
from particular periodica J .s . This latter file was particularly 
valuable for failure analysis when users cited articles which 
the service had failed to bring to their attention. 

Basic SDI service statis tn cs 

Various raw figures relating to the SDI service during the 
weekly operational service period (weeks Oil - 066) are 
given in Figures 1 and 2. 
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I III} I K p i mm ii« mmi^tmfm'mmrmi ■ i ii . m 

SDI SERVICE SUMMARY ~ DOCUMENT 



INPUT 



WEEK NO. 


DOCUMENTS 
INPUT 


AVERAGE 

INDEX terms/doc 


Oil 


199 


6.8 


012 


206 


7.h 


013 


267 


9.1 


Olit 


251 


9.0 


015 


250 


8.2 


016 


25^ 


9.0 


017 


2i*1 


8.8 


018 


2hh 


9.0 


019 


2h5 


6.8 


020 


196 


7.3 


021 


12i* 


7.6 


022 


223 


6.5 


023 


3hh 


7.6 


02i* 


237 


7.h 


025 


179 


7.7 


026 


375 


5.8 


027 


322 


5.h 


028 


2h6 


5.3 


029 


205 


5.3 


030 

w _y w 


215 




031 


3U0 


5.1 


032 


358 


5.0 


033 


3ir8 


5.2 



Appendix 



WEEK NO. 



03k 

035 
036 

037 
038 

039 
Oho 
041 

Ok2 

ok3 
okk 
ok5 
okS 
ohj 
ok8 
oh9 
050 

051 
052 

053 
05^* 
055 
056 



DOCUMENTS 
INPUT 

3^*7 
3kk 
321 
300 
3k6 
3^*1 
■ 3^*3 

3^*7 
268 

188 
206 
295 
273 
2k5 
2k3 
2k6 

2he 
195 

2i*1 

250 

2i*2 
3H 
275 



AVERAGE 

INDEX terms/doc 

5.2 

5»1 
5.2 
5.0 
^♦.9 
5.7 
5.7 
5.7 
6.0 

5.3 
6.1 
5.8 
6.2 

5.9 

6.5 
6.3 
6.0 

6.7 

6.3 

5.4 
5.8 
8.1 
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Appendix 



WEEK NO. 


DOCUMENTS 
INPUT 


AVERAGE 

INDEX TERMS/DOC 


057 


259 






7.4 


058 


293 






6.9 


059 


300 






6.6 


o6o 


266 






7.2 


06 1 


310 






7.0 


062 


232 






6.5 


063 


233 






7.2 


064 


400 


i 




7.2 


065 


246 






7.2 


066 


288 
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Chapter 7 



Delays in Scrvrice 



In setti.n/'^ up the service It was hoped to achieve a median 
delay of approximately ei^jlit days fi^oin receipt of the original 
journals t;o despatcli of the notif Lcat Lons» 

This reinai.ned a noble aspirat:ion that at no fc Line appeared 
in dan^^er of attalnmont* In fact, the overall delay does 
not appear to have di'opped at any time much below 16 days 
and at times reached 28 days* 

There are, of course, many reasons for the much longer delays, 
but undoubtedly l:hc f'act that, on ave!ca{>*e, at least six days 
were required for Iho computer part of the operation, l^e* 
from despatch of tapos to recei^pt of output, was a constant 
contributory factor* it should be remembered however, this 
f i.gure of six days Includes two non-workin(j days, i*e* 
Saturday and Sunday, and all the delays quoted are elapsed 
time, not workin^.^ days* 

The second major factor in 'the delay was the fixed weekly 
computer run* Any items miss Ln^^ one run were subject to 
another seven days delay before they could be input* The 
effect of this can be seen clearly in the graplis showing 
cumulative overall delay for different periods* (PLgs I-16) 

Tq achieve maximum cux^rency of the material it was of the 
highest importance to minimize delay between recei.pt of 
journals and the despatch of tapes to the computer* Tliis 
in turn entailed minimum delay in keyboardin/v the document 
input* However, the capacity of the sJ.ngle Flexowriter was 
barely sufficient to cope with demand and one of 1-ho hi/vhest 
priorities of the op<*ration was to keep the machine loaded 



7-1 



at ail limes. Thi.s iu turn Tiocossllalod keeping; a backlo^^' 
of inaLcriai and niili luted aj'^ai.nsL the keyboardinc of separate 
batches of documents. 

Fortunately the Plexowritcr was a f^alrly reliable machine 
and the perconta/vo of down- time was relati,vely small. 
However, any loss of i Lme was a serious matter and even 
routine sfMvicin/v became a probioin. Occasionally It was 
necessary to ask the sorvi.ce niochan i c~^ to postpone the service 
whon he arriveci at a particularly cri. ti.cal time. 

A3 thou/vJi liJo KlcxowtM tor was the most serious bottleneck in 
the sy3t(,Mn othej- machines wex'e an Important part oV the 
operat i.oih aii(J could cause trouble and delay. The most 
heavU.y used of* those was the Hank Xerox 9l4 copier used 
to copy tlie documents requix^ed Tor Lndexin^v Breakdown 
was Ifjss c^eri.ous here than wJ th tlie Flexowi^lter since it was 
not bo i 11/^ used to the same percenta/je of capacity. However, 
any bx^eakdown could cause delay, particularly at critical 
times in the weekly operation. 

The thi.x^d machine was a Multllith small offset iitho machine. 
Although thi.s was not very heavily loaded, tlie quality of 
work needed was hi(^h since it was used to print the card 
notifications. Thus, even x-^elatlvely minor blemishes could 
not easily be accepted and It was necessary to call in the 
service mechanic fairly often. This normally Involved a delay 
of a couple pf days and could, thei^efore, cause problems in 
spite of the f^reater spare capacity. 

There is no doubl that ruiui i.n{^v a weekly service which depends 
Jieavlly on the r(n!:ular operation of several macliines is not 
a comfortable occupation. Fortunately the people opera tin(j 
the machines became adept at fault dia^^ynosxs and temporary 
ropaix's, and in/^enious at devlsln/;; albernative methods of 
workin/j which enabled at least part of the operation to 
continue. » T ' 

P'luc lua tions in document intake 

The system was d('SM.vned on an estimate that tlie document 
input to the system would be approximately 2kO per week. 
This estimate was made cn the basis of an examination of 
the j ournal li teratui^e made In 1 9b7 • Kven assumin^^ that 
this estimate was accurate at the time, It was certainly 
out of date by I969 when the service began. However, in 
tlie first 15 weeks of operational service from November I968 
the aveiage input to tlie system was 235 documents per week. 
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In tho next ten weeks the input averaged 31 5 per week and 
i t was reaJ Lsod that Lhe low numbers in the earlier period 
had iKion cJuo to delay of material Crom the United States 
(Jun to iiidii.s trial disputes^ Hy reducing the number of index 
ti riM.s per dooiiiiiont and \>y «omo overtime work it was possible 
at tilings to acljiMvo a throughput oV 350 items in a w.eek 
hilt tlii.s was not surri.cient to prevent a build-up of a 
formidable !)acklor;« Naturally Ln these circumstances the 
overall delays became very larger Thus, in Fig 13 it can 
be seen that i'or this peri.od (weeks O33-O35) tlie median 
overall delay was 28 cJays. This period of heavy document 
Inflow last(»d until al»out Week 0^5 and it can be seen that 
in tho following tliroo weeks 0^6-0^48 tlie delay was reduced 
somewliat to 2k days. By Veeks 059-06l the. delay was down 
tp sixteen days. 

Summary 

Although considerable emphasis was placed on minimizing the 
delays of the system i. t was at no time possible to achieve 
tho currency that ve would have liked, for tlix^ee major 
reasons : 

1) One weekly computer run at a fixed time and at 
a distant location via an intermediary (DPC) 

2) Limited data prexjaratlon capacity on a single 
FlexQvri ter 

3) Large fluctuation in the supply of material for 
input 

It might be expected that it would be possible to single 
out some period of the 12 months operation as typical, 
but this is not easy without risking bias. However, the 
system did reach some kind of normality towards the end 
of the. 12 months and i.b may be supposed tliat the I6 days 
overall delay achieved in tliis period represents what the 
system was capable of in normal circumstances. , In some 
weeks tlie delay may have been two or three days less when 
all factors were operating favourably. 

In the light of actual circumstances the original aim of 
nine days delay was unrealistic. However, wj th a system 
allowing four days from receipt of documents to despatch 
of output and two days for despatch of notifications, none 
of which is unreasonable, an overall delay o I* nine days 
would appear to be acliievable. 
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Ki/i. i l)ol;iy from r..'C(.- i i, t .,|- joiirr uils to dospatcli of fapos 

Unn Nos. 033-o;j5 
No. of documents in samplo - 105 
No. of documents for -vhich date of receipt is known - 95 



Delay 


No. of 




Cum. 


(Days ) 


r\ %JL 




1 








? 








3 








^1 








5 








6 








7 








8 








9 


7 


7.J!< 




10 


15 


15.8 


23.2 


11 


17 


17.9 


h^.^ 


12 








13 








W( 


12 


12.6 


v3 . 7 


15 


8 




62 . 1 


16 


11 


1 1 .■> 


73.6 


17 


10 


10.5 


8/1.1 


18 


2 


2.1 


86.2 


19 








20 








21 


/| 




90 . h 


22 


3 


3 . 2 


9-J.C) 


23 


3 


.3.2 


96.8 


29 


2 


2.1 


')8 . 9 


32 


_1 


1 . 1 


100 




95 


100 
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*• • DcOuy rroin r(*c(M|)t of journal.s to (i(;.spa<:ch oV tapes 

I?un Nos* 0^f6-0/|8 
No* or documents in sample - 79 
No. or documents for which date of receipt is known - 65 



Do J ay 
(Days) 


No . 0 (■ 

(iocs 




Cum . 


1 








2 


1 


1.5 






6 


y.2 


10.7 




10 


15.^ 


26.. 1 


5 






26.1 


6 






26. 1 


7 


17 


26. 1 


52.2 


8 


12 


18.5 


70.7 


9 


12 


18.5 


89.2 


10 


3 


4.6 


93.8 


1 1 


3 




98.4 


12 






98.4 


13 






98.4 


16 


1 


1.5 


99.9 




65 


100 
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Pig, 5 Delay from receipt of journals to despatch of tapes 



Run Nos. 
No» oT documents 
No» of documents for which 



Delay No. oV 

'(days) docs 



1 


2 


2 


i| 


3 


5 


1* 


9 


5 




6 




7 


ii 


a 




9 


6 


10 


8 


11 


15 


12 




13 




li» 




^5 




i6 




21 




22 




23 




28 





70 



059 - 06i 
in sample 88 

date of receij- is knoim 70 





Cunt. 




2.9 


2.9 


5.8 


8.7 


7.1 


15.8 


12.9 


28.7 


- 


28.7 


- 


28.7 


5.8 


34.5 


lO.O- 


44.5 


8.6 


53.1 


1 1 .4 


64.5 


21 ,1* 


85.9 




85.9 




85.9 


5.8 


91 .7 


1.4 


93.1 


1.4 


94.5 


1.4 


95.9 


1.4 


97.3 


1.4 


98.7 


1.4 


100. 1 



100 
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Fig. 7 SDI Service Delay - 

Computer 

Runs 031-0^5 

Delay between No. of ^ 

despatch of tapes runs 
& receipt of 
output (days) 



1 






2 






3 






U 






5 


5 


33-3 


6 


6 


^0.0 


7 


1 


6.7 


8 






9 






10 


2 
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Fig.' 8 SDI Service Delay - 
Computer 

•Runs 0^16-060 

Delay between No. of % 
despatch of tapes runs 
& receipt of 
output (days ) 



1 






2 






3 






U 






5 


3 


20.0 


6 


5 


33.3 


7 




26.7 


8 






9 


1 


6.7 


10 


1 


6.7 
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Pig. 10 SDI Service Delay - 

Despatch of Notification 



Fig. 1 1 SDI Service Delay - 

Despatch of Notification 



Runs 031-0/^5 Runs046-060 



Delay between No 


. of 




Delay between 


No. of 




receipt ot output 


runs 




receipt of output 


runs 


& despatch of 






& despatch of 






notifications 






notifications 






(days) 






(days) 






1 


2 


13.3 


1 


1 


6.7 


2 


Z 


13.3 


2 


1 


6.7 


3 


1 


6.7 


3 




26.7 


U ' 


2 


13.3 


l4 


3 


20.0 


5 


3 


20.0 


5 


2 


13.3 


6 


2 


13.3 


6 


1 


6.7 


7 






7 






8 


3 


20.0 


8 


1 


6.7 


9 






9 


2 


13.3 
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\* \ {\. I h 'Ini.sil <lcl;iy f i om i » of' j<inrii;tl.s Lo (l(-.s]j;i i ch <? P no I.jM* i (*-a ti ons 

Kun Nos. 033-035 
No. of documents in sample -105 
No. of flocuiiKuits for vhlch date of receipt is known - 95 



Delay 


No. of 




Cum . 


(days) 


docs 






20 








21 








22 


6 


6.3 


6.3 


23 


15 


15.8 


22.1 


2^ 


12 


12.6 


3'i.7 


25 


6 


6.3 


.'11.0 


26 






/ii .0 


27 


8 


8.'i 




28 


9 


9.5 


58.9 


29 


1 1 


11.5 


70.^1 


30 


9 

✓ 


9 . 5 


79 . 9 


31 




5.3 


85.2 


32 


1 


1 . 1 


86.3 


33 






86.3 


3'i 




/J.2 


90.5 


35 


3 


3.2 


93.7 


36 






93.7 


37 


3 


3.2 


96.9 


^12 


2 


2.1 


99.0 


ii6 


1 


1 . 1 


100. 1 




95 


100 
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''"i/i* 1 "J" Total (Jcluy fi'oiii r-ccc-ip C of .journals to despatch of notification 



Run Nos. Oli6-Olt8 





No. of documents in 


sample - 79 




of documents for which date of receipt is 


Do lay 
(days) 


No. of % 

docs 


C um . 
% 


16 


_ 




17 


5 7.7 


7.7 


18 


3 ^1.6 


120 


19 


_ 


12.3 


20 


— _ 


12.3 


21 


5 7-7 


20. 0 


22 




20.0 


23 


8 12.3 


32.3 


2/» 


10 15./J 


'17.7 


25 


3 '4.6 


52.3 


26 


5 7.7 


60.0 


27 


i6 2ii.6 


Qli . 6 


28 


9- 13.8 


98.^1 


29 




98 . li 


30 


1 1.5 


99.9 




65 100 
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Total (i(-]ay Vvn m rocQUJt oV .journals to <l(-spaLch oT notirioation 

nun Nos. 059-061 
No. of; documents in sample - 88 
No. or documents for whicli date of receipt is known 70 



Delay 




4. 


Cum. 


(days) 


docs 




8 


2 


'> 0 
^ •y 




9 




1 /i 
1 • 't 


A 0 
/| .3 


10 






10.1 


1 1 


6 


8 6 


1 '7 


12 






/ij . u 




"1 




0 U )i 




7 


in n 


jn . ^ 


15 


6 


R 6 
t> • u 


'1 J . 0 


16, 


/i 




'10.0 


17 


8 


11.^ 


ou • .c 


18 


1 "5 


PI A 
<v 1 • t 




19 


2 




ft 0 


20 








21 


2 


2.9 


87.2 


22 




1 


88.6 


23 




1./| 


90.0 


2ll 




1./| 


95.7 


28 




1 


95.7 


29 




l.M 


97.1 


33 




1 Jl 


98.5 


35 






99.9 
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Chapter 8 



COMPUTER SERVICE FOR THE SDI INVESTIGATION 



The computer aspects of the Investigation were undoubtedly 
the least satisTac tory • 

It would be possible to go into very considerable detail 
but the main contributing factors may be fairly quickly 
summarised • 

1. The programs written by English Electric Leo Marconi 
were not fully tested before being put into use for the 
Investigation. This was not the fault of EELM since the 
large amount of data required for full validation was not 
available at the time. 

2. In view of the setting up of the Documentation 
Processing Centre it was decided that this body should be 
given responsibility for providing the SDI Computer Service 
to the Inves tigation. However only one person in the 
organisation had sufficient experience of KDF9 programming 
to be able to deal with the EELM programs. Since Mr Price 
as head of DPC, had many other responsibilities, he was 
unable to spend sufficient time on dealing with the frequent 
problems involved in setting up a new service using somewhat 
untested programs. 

3* DPC had no computer of its own but used time on the KDF9 
machine at the National Computing Centre. The availability 
of time on this machine, particularly at short notice as 
required for program modification and testing, was low. 

4. During the course of the Investigation the NCC KDFy was 
sold and it was necessary for the SDI work to be transferred 
to the machine at Salford University. Here again the general 
shortage of available time was a problem. 
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5* Poor 
tapes and 
program • 



housekeeping on a few occasions led to loss of 
in one case to loss of the latest edition of a 



6. Limited availability of time for Mr Price to write 
program modifications combined with the lack of program 
testing time on the computer led to accumulation of large 
amounts of testing with consequent likelihood of failure 
of the mn and consequent further delay. 

7. Programming effort was inadequate for maintaining the 
rcgu^.ar SDI service. Provision of such additional facilities 
and analytical programs as were desirable for an investigation 
of this type becaune virtually out of the question. 



The facilities available in the Investigation were therefore 
fairly limited. These were as follows:- 

1) Input of subject-interest profiles having a maximum 
size of approx 100 terms. (For much of the time, 
until the limit was relaxed, profiles with more than 
approx 50 terms had to be input in two portions on 
successive, normally weekly, runs.) 

2) Amendment or deletion of existing profiles. 

3) Printout of profile file in full or for particular 
profiles as required (the latter facility only 
became available later in the Investigation). 

k) Input of document records. 

5) Printout of documents in subject categories according 
to subject headings assigned to each document. 

6) Matching of document file against profile file. 

7) Printout of the documents matching each profile, 
either the document numbers only (General Matching 
Output) or full document details (Additional 
Matching Output). 

8) Printout, showing, for each profile, all the documents 
with which ±t had at least one descriptor in common 
and the actual descriptor ( s ) . (Special Matching 
Output ) . 

9) Input of descriptor terms to form or add to the 
Descriptor File. 

10) Printout of Descriptor File. 
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With thw- * facilities it was possible to provide the SDI 
service to participants in the Investigation* However, 
certain additional facilities were badly needed. These 
included 

1) Printout of Descriptor File showing, for each 
descriptor, the profiles in which it was used* 
A great deal of effort was Ln fact expended on 
this, including changing all the 600 or so profile 
numbers. However the only result was that the 
computer used one set of numbers and every other 
system a different one, so that the profile 
identifications had to be constantly converted 
manually by the lEE staff, from the one numbering 
system to the other. 

2) Usage of descriptors by documents. This information 
was required to allow analysis of indexing practice 
and adjustment. This facility was in fact available 
but there was doubt as to the method used to 
generate counts of descriptor use. 

3) Deletion of unwanted descriptors from the Descriptor 
File. This facility was supposed to be available 
but on the occasions it was used a' number of deleted 
descriptors continued to appear on the file. The 
underlying fault was never diagnosed and therefore 
use of this facility was discontinued. 

The effects of the deficiencies of the service wore 
of course felt in every aspect of the Investigation. 
As a result the profiles were of a lower quality and 
less than adequately tested before input, the weekly 
service was liable to breakdown and delay, fault 
diagnosis took large amounts of management time and 
profile analysis was more laborious than necessary. 
The special analytical programs which had been 
envisaged originally and which could have helped 
materially in the general axialysis of the system 
became quite unrealistic in the general context of 
the computer and programming facilities. Thus, 
all in all, it must be said that the actual service 
provided to users was considerably and dele teriously 
affected by the computer service- and, insofar as the 
Investigation hinged directly on the evaluation of 
this service, the results of the Investigation may 
have been seriously affected. Probably the most 
serious consequence, however, was tho nocossLty for 
the staff of the Investigation to devote alrnosc 
their energies to overcomLng the ocmputer service 
deficiencies and simply maintaining a regular weekly 
service. This necessary concojitratlon on the 
running of the service could not but have an un- 
welcome effect on the investigational aspects of 
the project. 
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In order to clear aoino of the px^oblems of error reports, 
a iiieetJ.n^r vas arranf>od between Mrs. PeJidJebury or DPC, 
P. Cla^uo of lEE, and K, AspinalJ of ERLM vho had been 
responsible^ for <:ho F>ro(;rains • Most of the error report 
probi(»iiis were c loar(»(i up, together with the forin of lo^^ic 
statement acceptable* in profilos, Th<> input J-estriction 
or a inaxiniuiii or 2iVi^ characters per field was queried, 
and Mr-, AspinaJ I ()!'/'<»r<»d to remove this limitation. 

Tlie possibili.ty of conroundin^j sin(ralar and plural in 
the thesaurus and oj' automatic amoiKJmejit or j)ro:r.ile terms 
in th<» li/,;ht of th<saurur> am(indments was discussed, tlv^ 
former bc-in/; thou^rin. likely but not the latter. As a 
result ol' thes<» diseussiojis it bc^caiiKi aj")pax'<Mit that the 
pro^^raiiis had b<M;n wri tten on an assumption that pi'ofilos 
woul(i bi} uniquely id<Mitified by tlie last tliree di^^its oV 
the pj^ofile 3iumb<»r. Wliile tliis arx-an^romcnt could have 
been adopted in numlxa-in/j the profiles this I'equiroment 
vas not communicated to TEE by En^jlisJi Electric and the 
actual numbers assi/vned did not conform iwxth this require- 
ment . 

J.t was thou^^ht by Mr. Aspinall tliat some throo weeks 
j>ro^i'rammin(i' effort would bo Jieeded to write a pro^xam 
to convert the recoj-ds from the exist:! n^v numb<*rs to 
a^rec^d numbers wi (.h tho last throe c\±(slti^ unicpie. OJie 
other way vas, or courses, for XEE to re-ijiput th<^ profiles 
wath clian^c^d iiumb(M-s. No imruodiate decision was takeji 
on this matter since aLl tlie possible repercussiojis on an 
alr(S-idy (^Ktremely ti^^ht lichedulo had to be c<)nsid<M-ed . 

Th(^ no(KJ lor aji ajialyticaJ printout to show the us<^ mad«j 
of any descriptor by ajiy px^ofile vas mentioJ)ed ajid 
Mr. Aspinall a^j^rcuMl to look into this. 

A moetin^r was JkvUI um ;il February, 1968 between OSTX, lEE 
and D]>C. The matching- of px-ofil^^s and documents J-equosted 
m tho January meeting- (roquirement (e) above) liad not 
yet been obtained. Mr. Price stated tJiat one dirricnlLy 
was tJiat tli^^ SDl pro^^rams accf^ptecJ docum<iJit input only in 
the ronii or paj>er Lap(^ and the (j^istln/^ tap<^s wer<* r;^A.l\ti(\ 
worn. A modiricati on to a(:c(i/>t ma^'-nr^tlc tap<i wou I d \-<'<pi i i'<» 
one iMan/w(^ek»s pro/ri aimnin/v time. « JKIC slUl had no luVarm- 
ation as to tlie documents actually accepted on rii<. and 
DPC orfeied to provide a document tape odi t to ^^jve this 
data. 
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FulJ .ir..> tract i.ons on tho .i.iij)uL data .l.i.iii.L ta t.i ony and foniiaL 
re(iui.r,;iiicMits wore Phially recc. j.vod rt-oiii DPC on |/| March 1 <)6H 
thus allowin,i IKE (.„ prepare papcj- tapes to an a/'rood 
specification. Fvom this date Llie Input error rate dropped 
stoopiy. Thi.s lact, together with the newly acquired know- 
ledge or the state P the <locun.ent Ci.Le (rron. the document 
tape edxtj, considerably eased tiie problem of profile 
tes tj.nr; nnd modi r.i ea tlon . In spite of all tlie computer 
dj.ll.Lcult.Les and consuquont s.-rious delay Ln obtaining 
test matching output, it was s t,i 1.1 the aim to start the 
oxperxmental s.-rvic- period (PJiase /| ) in April 196K. llou- 
ever, this became completely impossible wlien in early 
Marclj the wrong profile lapdat.- tapes were input by tlie 
NCC operators. Th<- resulting confusion, compounded by 
tlie poor record keeping which had been primarily rc opens ibl^^ 
continued thronghoni MarcJi and most of Ai>r.! 1 and during 
thj.s time no matching output con Id be obtai.M.-d or new pro- 
fi-le data input. The problem of clearing th<-se troubles 
was made worse by tl.<: lack of available time en tin- NCC 
machj-ne, a waiting lime oT one w<M>k being comlnoii at this 
time. 



Alter almost two montijs, during which «o usable output 
was recej-ved, a successful run was made on 30 April, 1968. 
This run finally iin-orpora ted tlu- profile modiJ-icv tiou? 
which had been accumulating ovc-r tin. whole of these two 
months and produced matciiing output. However, in a number 
of cases more tJian .,nu modification had been made to a 
profile m that tj.m.-. Wiiore this had happen.:d the progrr^ms 
accepted only one modificat Ion. and produced output on tlie 
basjs of that modification and ignored any otiier. Tt vas 
o£ course, unfortunate that the clioice of modification va4 
random and that th<.ie was no way of discoveri.ng which one' • 
had been accepted, short of checking a printout of the 
profile file. 

On 22 May 1 968 TEE queried t),.- status of the pro -rams for 
converting the profJio numbers from the exls t Lng*'unes 
to one with unique last three digits. UPC reported that 
this prog,-am might work given optimum conditions l.ut that 
modificauions were i,<;ing made to take care of Jikeiv 
problems. 



At: this? timo J.EK vore concerned about the layout of various 
printouts o.^v Bullotin output and General Matching Output. 
These had boon spec:i fled to DPC on 5 March 1968 but no 
action liad been taken, 

Throu^;hout May and .Fune tlie computer service improved very 
iiiark<»dly both in Trc-quency, reliability and speed of turn- 
round. TJii.s was .ju>t as weJ 1 since Phase ^1 in which users 
rocoJ-ved sample sor\ Lee had now started. Massive effort 
was bein/7 <I<-voted t.o <5S tab 1. i shln^.^ the exact (;ditdon of the 
profiicTS acUiaJly on !MJe, (^valaating the test matching 
runs, modjf'ying as ii<'c<»ssary , r<»-modi f y.i ng and ro-testing, 
'and finally pass.i.nr; tJie px'ofLles for service. Even at 
tli.is sta/C'-, Iiowvc'j-, thiu^o remained uncertainty as to the 
oxaci J imitations on the sii^.e o !' particular parts of records 
and these- limits were transgressed occasionallj'' as profile 
recoi'ds Increased J n size . 

Full i nroi-matloii on the size limits for various parts of 
the records was not received until ^July 1968. Computer 
service duriiig this period con t i.nued to be fairly satis- 
factory and It Wfis possible to provide an ex])erimcntal 
sorvjce to most usc»rs. Tfowev<»x', a number of" inadequacies- 
in the pro/creims continued to h<y discovered and to cause 
trouble. In particular, modirications to some profiles 
were being rejected Cor no easily apparent j-eason. This 
resulted In a core of untested and unimproved profiles 
which could not be put into experimental service. 

On 10 September l9(yH the status of the ro-numberlng prog- 
rams was that they appeared to work but that the print program 
was nut yet runctJoniiig and therefore there was no easy 
vay of cliecking th«- programs. 

For the Last two v/<M-ks of September, no output x/as received 
from DPC. This turned out to be due to Mrs. Pendlobury * s' 
absence and tho fact that nobody else liad been deputed to 
deal with the work. On h<-r r<-turn the accumulated tapes 
wore x-\m but a falJure occui^red for an ol>s<:ure reason whlcli 
could not be dealt with in Mr. Price's absence. These 
l:apes were finally i-uri on 8 October - giving a total of a 
montii in wliich no output had been received from DPC. 

Since the start of the operatiional period (phase 5) was 
scheduled for ! N^ovember 1968, there was considerable 
anxiety about the computer service. 
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A ictLei- rroiii .I.EE dji.i:<.d 7 Octobor suniiiiao.i-.ed the roquiro- 
iiients to he met belore 1st November, These .included: 

a) Re-numberin^j proci^ams to be coiiiplotod ajid checked, and 
the re-nuiiibered files to be in operation. 

b) Bulletin output to be produced -on litlio mastcTs as 
a tost. 

c) AssTiranco of a i o;-u.Lar we<«kly coiiiputoj- sorvici> w;i fch a 
rapid turn-rouiHl t:imo. 

Til a telepliojio coiiv<Tsa t: i on on ;i2nd Octobc-r 1908, Mi-. Prjc<! 
promised that tlio r<-nuiiib(;rin^^ would be carri.ed out as 
soon as possible and no later than Llio end oP Ociobor. It 
was agreed thai work on tlils sliould be f-lvon priority over 
tliat on residual pro^ri^ain errors. IRK promised to seiid 
punched paper t£ipe i ncorpora tin^^- the necessary re-nuiiibo r Luf 
data lor a quarter of the profiles. These w<«rc despatclied 
the same day. 

At this period (October 1968) shorta^je of computer time 
became acute and this situation was not helped by tlje fact 
that a larfi-e number of modif.i cat.i ons had no« accumulated. 
As a result of this shorta^^e of comj)uter t,im(>, profile 
modifications despatched on l6th Octobor were not matched 
untxl the 29th, by t-i.o same run used for testing the modified 
programs and tlie re-numboring program. 

In spite of aJ J tJie computer probjems it was felt to be of 
over-ridmg imporun.ice to .start tlie wooklv SJJI S(>rvlce to- 
users as soon as po.^siblo, and this was done at the be.-inning 
of November 1 968 . All in ail, thv minimum service of provld- 
xng the matcliing output each week was maintained with 
varying turn-round .hrlays. However, attempts to obtain 
xmprovemcnts on thi> very basic level of service wore un- 
fruitful. 

By th- end of November 1 968 no furtlier progress had been 
made n profile re^numberlng and various program modifications 
largely, as always, througli lack of computer time and the 
.'absence of Mr. Pric<-, the only person at DPC abl,. Lo J.aj.dle 
the SDI programs. His visit to the USA meant th,-. t tio 
progress could be made for the whole of JJecembtM-. Th.is, of 
course, entailed at least one montli's cxi-rn delay to tii<- 
SJDI Investigation's proposed schedule since i I. vas agreed 
that Phase 5 v^jpera tional service period) comI.I not ovPflcially 
be considered to have start<id ur-til tJie re-number In,-- pror- 
ramo and the various other program modif icat j ens Jiad beon 
completod . 
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Tn conversation witit Mr. Price on 7th January 15)69 it 
was Icnrnori that tlic ro-nuinborine program was considered 
t.o »)(. uhoul, two- l;h i Ids operational.. The ma in reason Toi- 
I.M0 (J<-I;iy in its oompLotion was turn-round tliiio of about 
a wo(,k lor p.rotvram l,ostin/i-. The other pror;raiu modirications 
wore l)oj.nf; Input and tested on the same runs. 

At this stacc the NCC KDF9 was being sold to the National 
Physical Laboratory and negotiations to use the SaJford 
Unj.v<>rsity KDF9 were proceeding. However it was tliought 
tliat the NCC mac]iinc would be aval l.able for the SDI work 
Until June/July 1969. 

On 13 February i.t was learned that NCC had lost the 
latest version of tJio program tost tape, witli tiie result 
that xt was having (o be rebuilt from an earlier version 
to xnclude all subs<«queht amendments. Two weeks had been 
spent on this cind another two were expected to be necessary. 
All work on the re-numbering program and program modifica- 
txon held to be delayed while the tapes were recohs tituted . 
No xndxcatxon could be given as to when these modifications 
would be complete. 

At this stage tlH« earliest possible date for start of 
Phase 5 was 1 April 1969. As even this date became less 
and less likely the T.EE became very concerned and made 
very great efforts 1.0 clarify the possition and to seek, 
xn consultation with OSTI and DPC, ways to minimize 
lurther delays. (Copies of relevant memoranda, letters 
and mxnutes of a meeting of 10th April are shown in 
Appendices 8A, B,0, & D). 

Further meetings between lEE, DPC and OSTI, arranged for 
2H Apr..l and 28 April 1969, were cancelled at the last 
minute . 

On 30 April 1969, lEE received printout of a re-numbered 
profxle fxle xncorpo rating profiles for which data had been 
provxded by lEE on 22 October, 1968 and 3 January 1969. 
Several days later a second printout arrived showing the 
use made of the Descriptor File by profiles witliin the 
range covered by tlie tapes sent to DPC in October 1968. 

In the light of these successful printouts it appeared 
reasonable to expect that a completion of the i-e-number Lnr 
of the complete file would be no more than a few weeks away. 
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A numbor of chan£;os to profile number.-i had occurred since 
tho ori^rlnal tapes were sent, and lEE offered to produce 
:inuiiodiately additional tapes to incorporate these" data. 
However it was decided, in consultation with DPC, to defer 
these unLi.1. a complcLe printout had- been obtained and 
chocked on t}ie existing profilo file. 

TroubJ.(5 stlLl continued Lo occur in updating; profiles, 
ieadxn;- to the rejection of apparently perfectly sound 
modi Pica Liions. Tho reason for such faults could only be 
dxscovet-ed by som(;one with an Intimate knowJodge of the 
programs, i.e. Mr. Price, and his frequent absences from 
Mancli(>s<,er entailed considerable delay in discovoring and 
correcting sucli faults. It was particularly irritatini"- 
fchat many of these liad been talcen care of in the program 
modx Pica lions vliicli had been in process of being tested 
for approximately sJ x months. . 

On ^ JiiLy, Mr. Price was again asked about the re-numbering 
progi-aiii. The three types of error discovered had all been 
accounted ^or and= completion of the work appeared to be 
close. The ma .in problem was computer time for testing 
since a run required three hours. 

In the same conversation lEE mentioned the small layout 
changes required .for Addi tional Matching Output In order to 
discovej- any possible snags before the computer stationery 
was ordered for this. lEE requirements appeared straight- 
forward to Mr. Price. lEE offered to send full details. 

On 22nd July, Mr. Price was again approached about Add- 
xtxonaJ Matching printout in the light of his letter of 
15 July. The noad for speedy action on this was agreed 
and xn the light of his imminent departure on holiday for 
fxvo weeks he af-reed to make the minimun! adjustment immed- 
xately. He consider-od fuat the re-numbering program, was 
now workxng perfectly and the lack of progress was 
ascribed to a machine fault. 

For tho last week in July and the first two In Aufmst,- 
scrxous trouble occurred .in running the SDI programs, 
with tlie result that no output at all was received during 
thas periods The interrupted service was resumed on 
18 Augut,t and major efforts wexe made to clear the backlog. 



Tfj(* viiw (iV 11 SoptcMiibor iy69 raJ^Ied as a rosult oT a 
lo/^ic sLalcMruMil, (irror. Tills iauit liad boei) known Tor 
soiiie t.iiiK^ and ;\ pror.ram modiC i oa t.i on ha(J bcMiri written 
sovornl iiuniUis (^'iidicM^. It was not iininodln t>ely known 
whotiHM' Lhi.s in(jdi.f'i (N'lt.Lun had boon tested and .incorpoj:'ai;ed 
or winaiM^r as a result of reeont troubles an earlier version 
oi' th(j pro^j:t%'iin was being used. 

At this time a(ld L t.i onal problems began to occur with 

thu changoov(^r Vx'oiu NCC to Sa 1 To rd University. It was 

found not possxl^le to print out the General Matching 

(the nornial no t Lflea tion to subscribers) in a format to 

fit the computer r stationery so tliis was provided temporarily 

on plain paper. 

On '3 October Mr*. Pr.i ce was asked by lEE about the Additional 
Matching; Output. Th Ls had not worked yet, possibly due to 
inachiJiu faults. Tlie urgency of this output for the various 
studies and the need to restore the facility lor printing 
out the updat(Hl version oT pjofiles after modification was 
stressed. This latter facility had been civailable up to 
the troubles or Au/>ust but had been lost since then. This 
left no way oV checking the profile updating. The need 
for a progress report on the re-numbering program was also 
mentioned. 

Thus at the start of the weekly service in November 1969 
the situation could be sumarized as follows 

1 ) Time on the Salford machine was inadequate except 
lor tlio regu.lar SDI weekly run and only then ±f no 
problems occurred or special requirements were 
speciried. 

2) No additional time was available for progx^am testing 
or for re -run in tlie ev(;nt of failure. 

3) To minimize the danger of non-arrival of tapes by 
the scheduled run time, large allowances had to be 
made, thus causing delay in the service, ie tapes 
sent by lEE on Friday afternoon were run >>y DPC on 
Tuesday evening (four days later). 

h) Twenty months after the requirement for re-numberlng 
was accepted by DPC, this work had not been completed. 
Thus during this whole period the SJ3I s tafi^ used two 
separate numbering systems for profiles, with all 
the waste of time and unnecessary complication that 
this entailed. 
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. ')) Annlyc^is hc'i.scd on u.sci o (* narticular <l(vsci\Lp lions by 
pr'oPi Ics Wc'is slLl I J iiipossJ b 1 o . 

6) Pro^^nun faaJl-.s causin/r j^ojf cti.ons of pj^oTllo amend- 
incn(:s wore s i 11 active ion^^ after the causes had 
b(Mjn detected and in soino cases pro^^ram modirications 
w rJ (; ten • 

7) I^rintout of updated profiies had not been restored 
af t(ir eariier troubles . 

8) Additional Motcliin/'- Output required urexuitly for 
several studi(^s was still not available in the 
format requ.i red. 

9) Chan^^(M)ver from the NCC KDF9 to the Salford KDP9 
was throwin{^ up piob.leiiis even in the normal oper- 
ation of the SDX service in spite of the fact that 
this chan{^eov(^r had been known about for six months- 
at least. 

Summa ry 

The f()ree:oin^j: recitnl of the difficulties encoiaitered in 
the comput(^.r service* is necessarily lon^ and complex. It 
is perhaps useful -tlK^refore to summarize the facilities 
tiiat were. availabJ e during* -the Invt^s tigation:- 

1) Input of subjcct-infcer<vs t profiles havin^r a maximum 
size of approx. 100 terms. (For mucii of the time, 
until the limit was relaxed, profiles with more than 
approx. 50 t(^rms had to be input in two portions on 
successive, normally we )kly, runs) 

2) Ajnendmont or deletion of existing profiles. 

3) Printout of profile fil(^ in full or for particular 
profiles as required (tiie latter facility only became 
available later in the Investi^jation) 

h) Input of document records, 

5 ) Printout of documents in subject cat(v';o eies acco rd- 
ing; to subject headinf^s assi^^nod to each docuaieiits. 

6) Match Ln{^ of document file against profile) file. 

7) Printout of tiio documents matching; each profile, 
either the document niombers only ^Gener-al Matching- 
Output) or full document details (Additional Matching 
Output). 
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8) Printout shovi.n^r, lor each profile, ail the dociaments 
wLLh wliich 11 had at least one desei-iptor in common 
and the aetual descriptor (s ) . (Special Matching 
OuLpiJ L) . 

9) Lnput oi; cicscriptor terms Lo form or a<Jd to the 
Descrlptoj" Ki Lo. 

lO) Printout of Ih^scriptor I^LIe. 

With lh<Jse Taci I.i t i <»5> it was possible to provide tlio SDI 
sorv.ic(^ to partici|);ui ts in th<^ Invos ii{,vation . However 
certain additional laciiiilcs wore badly neodod. Tliese 
inc.1 udecJ : - 

1) Printout of l)^^scriptoJ* File showing:, j'or each 
descriptor, the profii(^s in which it was used. A 
C-reat deal of efTort was. In fact expended on this, 
inciudin/r chan/^in/j all tlio 600 or so profile numbers. 
However the onJ.y result was that the computer used 
one set of numbers and ovciry other system a different 
one, so that the profile identifications had to be 
constantly converted manually by the lEE staff, 

from the one numberin^r system to the other. 

2) Usage of descriptors by documents. This information 
was requhred to allow analysis of indexing' practice 
and adjustmcMit. This facility was in fact available 
but there was doubt as to the method used to generate 
counts of descriptox" use. 

3) Deletion of unwanted descriptors from the Descriptor 
File. This facrlity was supposed to bo available but 
on the occasions it was :used a number of deleted 
descriptions continued to appear on the file. The 
underlying fault wn* never diagnosed and therefore 
use of this facility was discontinued. 

Tlie effects of the deficiencies of tlio service were 
of course felt in every aspect of the Investigation. 
As a result the profiJes w<;re of a Jower quality 
and less adecpja t(vl.y tes te(J before input, the weekly 
service was lKabl<5 to brcrakdown and diilay, fault 
diagnosis took iarr/e amounts of management t.i me and 
profile analysis was more Jaborious than ruMu^ssary. 
The special analytical pr<^grams which had b<ien 
envJsaged orJginally and which couicJ have helped 
materially in the general analysis of the system 
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l)coam«j <|nlt<- uiir(s-iJ..i.st;i.c in l.lio /.7ojior;»l t<ont(>xi. 

ol" tho coiiipu(,cr and prof^raiiiiiii.nf; Pac i. J i. ib i.cs . 

Thus, all. in al .1 , ,i.t iniisC bo sa.i d tJiab, tho ac (.uaJ. 

.s(>rvic(; piovid(id (;o usora was consJ.d(M>ul)ly and 

dcsJefcoriously ofloctod by tho comi.utor .S(>rvi.c<- and, 

xn so Tar as tho Invos ti<^aciori hinfi-od directly 

on tho evaluation this so.rvioo, i;-..- results oC 

tho Inves titration may havo boon serious, jy afX-cr totl. 

I'robably tho most sc^rious cbnsciuonce , liowovorj was 

tho necessity for the stafr of the Invostic;at j on 

to devote almdst all their enorg:ies to ovcrcomJnr " 

tfie computer service deficiencies and simpl.y 

maintaining- a reguiar veokly service. Tliis noccssnry 

concentration on the runnintv of the service could 

not but hav<« an unweXcoiiie ofCoct on the inves tirational 

aspects of the pr'oject. 
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Chapter 9 
PROFILE ANALYSIS AND MODIFICATION 



By 3 Novombor 1967, ::02 profiles had been compiled. This 
was good pro{;ro3S, par ticularly bearing in mind the lack 
of a full complemenl; of staff from the start of the Investi- 
gation in July. By March I968 virtually all of the 570 
individual user prof :i les had been compiled. 

Apart from the central dif f icuJ ty of learning how to render 
often complex research interests in an SDI profile, the 
initial task of profile construction was complicated by;- 

1) A thesaurus which had to be developed con- 
currently w:i th profile construction. Thus, 
as the thesaurus expanded to include terms 
needed for new profiles and for document 
indexing the earlier profiles required to be 
updated to take account of the expanded 
vocabulary. 

2) The inability of the computer service to allow 
testing of draft profilos and to give 
experience of the interplay between dociament 
indexing ajid profile compilation. Only when 
the bulk of the profiles had been compiled 
did this data become available. 



One of the major problems in profile analysis and modifica- 
tion for a long time was that no record was obtainable 
showing which of the documents inpuc to the test collection 
had in fact been accc'pted on file and were therefore 
available for matching. It was not imtil the end of 
February I968 that tJiis data became available. 
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The draft [)rofilos were tesUod by inatchin{j them against a 
test collection of , some 700 documents which had been indexed 
and input to the computer in the period September 1967 • 
February 1968* Here again the thesaurus had developed quite 
consxderably in the interval between the indexing of the 
first documents and the last ones, and the indexing was not 
therefore in any way xaniform. 

However the purpose of these first tests was not to establish 
accurate performance figures but to gain some idea of how 
well the profile was performing and what modifications were 
necessary before it could be considered ready as a basis 
for experimental service. 

Test method 

The test collection of docujnents were selected from readily 
available core jourmils in electronics e.g. IEEE Transactions, 
lEE Proceedings, Electronics Letters, J. Applied Physics and 
some of the Russian cover-to-cover translations in this field. 
It therefore covered a largo number of the topics in the 
profiles but no attempt was made to ensure a representative 
sample to cover all topics. 

Each article was assigned to up to three out- of approximately 
15 subject headings and lists were typed and duplicated of 
the documents assigned to each subject heading. 

Sets of subject lists appropriate to the user^t; interests 
were then sent to him with a request that he should mark with 
a tick any articles of relevance and with a question-mark any 
of possible interest but whose relevance could not be ascertained 
from the information given. 

This was our first taste of serious misunderstanding since, 
though our covering Jotter expressly stated that this was 
for test purposes and was in no way the SDI Service, a number 
,of users commented on the "servjce". Some were highly critical 
as was only to be expected since in many cases it was likely 
that out of 200 articles only one or even none would be relevajit. 
Mo re surprisingly some users were happy with what they had 
r ived. 

The marked dociinients on the lists were taken to be the 
relevant documents in the system. Those ticked were taken to 
be the relevance »1» documents and those with a question mark 
as approximating to relevance »2*. All unmarked items were 
considered as not reJevant. 
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OomparLson of these wi.Lh tJio document se3octod by the 
prof lies provided fi.{.:ures Tor Recall 1 (relevance 
documents and relevance documents combined), and 

Precision (relevance <1< and «2< combined). Any docunionts 
selected by the profile but for which we had no relevance 
assessment (i.e. they had not been included in the document 
lists) were i/jnored in arriving at precision f injures. 

Fibres achieved at this stage were (20 per cent sample - 
universities ) : 

Recall 1 T^ecall l/2 Precision 

First analysis and modification 

After analysis and modification the figures had been 
raised to: 

Recall 1 Recall l/2 Precision 

72i> 58<jL 6l^ 

It was considered, on the first test results, that, though 
there was obviously room for improvement, particular attention 
should be paid to improving Recall and in particular 
Recall of relevance *1« documents since these represented 
documents of real interest. The reason for concentrating 
on Recall was that it was felt an initially somewhat over- 
broad profile would show its faults in the Precision 
returns and could bo adjusted. A too-narrow profile on the 
other hand would not be self -monitoring in the same way 
since it would not bo so easy to decide what was being missed. 
In addition, of course, the poor Recall figures were in 
some part due to changes in the thesaurus since the 
profiles were compil<5d. 

As can be seen from tlie figures a great improvement in 
Recall was achieved while holding the Precision at the 
same level. This was not the maximum improvement possible 
s:i nco a level of perf ormance equivalent to approximately 
60% Precision and 60% Recall had been set as the pass Line 
and profiles reaching this level at Test I wore not, in 
general modified further. Some five of the forty profiles 
in the sample were of this type. 
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To somo ext.ptit tlio xinprovoniont in performance sliowii by the 
ri{^ures is illusoi7 .since at this time the analysts knew 
which documents were relevant to each profile and were able 
to modx.fy to ensure that these were retrieved. However 
they were instructed not to modify in this narrow way but 
to ensure tlxat the modifications were consonant with tlie 
users* original stat(»d requirements. While it was 
expected that the poj'formance figures obtained in this 
second test would bo artificially hi{^h, some of t)ie increased 
performance and the shift toward better Recall would be 
maintained. 

As mentioned earlier the level of perfonnanco required of 
a profile before It could bo considered ready ('or the 
experimental .service* was tljat Lhc f i(jures for percenta{jo 
Recall and Precision sliould, when added, total more theui 
120. For example 60 per cent J^ecalJ and per cent 
Precision or 100 per cent Recall and 20 per cent precision. 
However, the dan^;cr of extreme fi^jures was realised and an 
attempt was made to bju.ne- each fi/>ure more into the middle 
of the range. Empha.sls was placed, as mentioned earlier, 
on Recall so that, though a profile giving 90^ Recall and 
30^ Precision would be accepted, one with the figures 
reversed would be modified in an attempt to increase Recall. 

To some extent, of course, the level of performance taken 
as acceptable was arbitrary and, thouglx based partly on 

earlier experience in the manual system and an idea of 

what users might con.sidor acceptable, was largely a way of 

deciding those profii.cs whicli most neodcd attention within 
the limited time available before the start of the 
experimental service . 

Experimental Sex^vice 

Some profiles were still unsatisfactory after modli*lca tion 
and second tes t and in some cases further modifications 
were, made and tested. However, it was not pos.sibie to 
bring all profiles up to the stated level of performance 
before they were used for experimental service. There 
were vai^lbus reasons Cor this. The test collection dxd 
not provide sufficient data for a number of profiles 
in the inoi-e fringe areas of the field and op 1 i irtizatiori 
on such data would have been unv/ise. Otlier proI'iI<is 
remained poor performers after all efforts to Improve tliem 
and it becajne necessary to seek additional information 
by putting them into experimental seirvice. 



I.'jy of the 181 unJ.vci'sity users recoived output Tor the 
first experimental service run. 

Average Precisioii fOx theso (20 per cent sample) was 62 
per cent, taking rricvance »1* and relevance documents. 
'^Figures calculated as avei^agc of ratios)* 

lyj of tJio 19*3 {.government users received output from the 
first exporimental run, and 109 out of 191 of the 
.indu-:trial users. * 

Huis, by the end of' Juno I.968 some 38J profil<»s had been 
put on f.ield, tost<Ml and had generated a first set of 
experimental notifications for their users* 

llio relevance returM.s for these notifications were monJ tored 
and action was taken to improve precision wlxore this 
appeared necessary and possible. 

At the snme time a considerable amount of work was bei^ig- 
done on compilation of new profiles and on testing and 
m.odification of profiles which had not yet been passed as 
satisfactory for exporimental service. To a large extent 
the delay in getting profiles into the experimental service 
was due not so much to inadequacies in the profile £ind the 
consequent need for modification but to delays in obtaining 
test output from the computer. These were due to a number 
of reasons including data preparation errors, occasional 
abortive runs, and dctlays in setting up a run. In addition 
the error reports wej-e somewhat cryptic, causijig difficulty 
and delay in locating and correcting the faults. 

However, major efforts were made to cleax^ the errors that 
had been preventing acceptance of original and modified 
profiles and the numl)er of profiles tested and ready for 
service grew quickly: 

Profiles ready for experimental service 





May 


. . ., 273 




May . , . . 


. . . 'Jlth 


6 


Jimo . 


' . . 379 


11 


Mtaic . 


. . . 399 


PA. 


June . 


. . . liho 


10 


.July . . . . 


. . . '190 


6 


Aufjust 


. . . 53'< 
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Pi:rVortt\^ii\ca In Expoci mental Sex*vicc 



In order to {^ain sonic idea or the Recall performeuice of 
the profiles, partia,! lls^s of the articles input to the 
first four series of experimental niaXchin^j runs wore 
sent to users vith a request that they should mark any Items 
which their profiles should have selected. 

The r(!.sults of these; sliowed Recall (for the same 20 per cent 
sample of university profiles) to be approxima teJ y 55 per oont 
while Precision was 7 1. per cent. Though sorae of the 
improved Recall aftcj- tost 2 liad been losfcj obviously a 
considerable rocil irnu-eci5>o iiacl boon achieved over test 1 
without any sacx'lfictj of Precision. 

As before, a .lev^l of acceptable performance similai* to the 
previous one was sot, though this time xhe dangers of very 
low figures for either Recall or Precision v/erc realised^ 
cind in genex^al a miaJmum figure of ^0 per cent I'or eithex* 
value was set. Once again particular attention wa-. paid 
to Recall r 

Preparation for opera tionei L service 

A target date of 1 November had now been set Cor acnuM^iicement 
of the weekly sex^vict; and time wsxs extremeJy sliort for 
carrying out all the testing, analysis and modifications 
desirable by that time. Consequen tJy it was only possible 
to analyse and modify obviously inadequate profiles and 
many acceptable proPiMes couJd not be given !thc small 
amount of attontdcri which m3ght have served to iinprove them 
considerably. 

The marked-up Jists began to be sent back by user.s about 
mid-August which, of course, left little time 1-or analvsis 
aiid modification before the start of the serv.ioe. Some 
px^ofilcs however had not been put into service until riux 3 
of the seven experimental runs. To obtain sufficient test 
dava it vat, necessary to send to these a.-ici's-, docurnont 
llsrs of runs 3-6 i.ncJusivo. In addition, for some 
profiles, tho !;<"st data fx^om ruiiii 1 - U had not been 
sufficient and it was necessary to send listings for runs 

5 and 6, In general wJiere there were fewer Uian |.w«^Lve 
known relevant docUiMt-nts for t:!»e test- on nuis J « A, 
addition?al listings Tor run^. 5 and 6 wore sent. Fiv- and 

6 listings were also sent to iwovs who:.^e pro riles had been 
modified as a result of the 1 - h analysis, Ln orchjr to 
assess bhe effect oi* tncse modifications* 
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Tho 'A) I i s i COM 1 (J HOC be- t>«*nt out on til inid-Oc tobcr 

•so t;h:iL iiiiicli of tho rasulttxryl anuLysis and modi f.i.ca t:i. oil 
could not Ixi dofu- b'-Torc the st*irt of tho weekly service 
in Novi'inbc r. 'fiius , i n the Tirs t month or so or re^juiar 
weekly service many users veris receiving service on 
profiles which wore as yet reiuxi.vely untested or known 
still to be unsafcisi'ac tor>'. This s tate of afrairs must, 
be attributed to lack of one analyst/indexer for the 
proceeding: twelve months of the lnves^igation and to 
delays In achieving a satisTactory computer service.- 
There is no reason to suppose that^ without these 
handicaps, vix^tually all profiles would not have been in 
a properly tested am J modified state by 1 Novc^mber* 

It must also be rciiic-iuberod that tlio experimental service 
pf»xioc!j ^>ri/]^Lnally planned to last six ihonths, in Tact 
onJy J as ted fi-om xMay 17 (when the Tirst notifications 
were- slespatched) to I xNovember, l.c* f ive-and-a-hair 
months, ?uid covered i>he main hoj iday period wherl 
Interaction with the user, vitaJ for prorile cvaluaticii, 
was most snbj/jct to delay. Tht* start or the regular 
service could have bi-en delayed to allow more tiirie Tor 
profile modification but it was felt i.'ssential to start- 
pro v Id ing Che serAdLeo which had been promised Tor so long^ 
since thei-^ i:^s a grave I'isk of the users becoming 
complt^tely disenciianted. This decision was not iaKt-n 
l)(^htly since tlie iinrvit^ible result was that effort needed 
for Pi '^f Lie testing and mod i.f ica tion was diverted to indexing 
and other activities rxssociate«i solely with provision 
f>f the s<»m'icc» Thf-^ rneant^ of course, that the Joss oT 
time dnr ing tho experimental service could not be made np 
by eijuLvalent time dux^ing the opex^atioual sftrvice. 

Weekly operational service 

Aft'»i ^hf* weekly serv^jce liad been runn:ing fox' a few weeks 
rlocumeMi! listings were agiiin j>.''nt out in an attempt to 
asses^ tin- x^ecalj of the sy stent. Performance figures roi* 
this run (Oi^i) were Precision = 70 P<^r cent (average of 
i-alios for lO'j profiles, i.e. approximately 20 oer cejit 
samp]*-); Recall = '>9 per cent (average of ratios for .707 
profi les ) . 

Some pro fil*- moil iTica { i «^>n was tJone o n the indivi dual 
pro! i J OS where the Reca:.!! figures secined to be extremely 
poor, but It was decided that xoi varioxjs reasons no overall 
analysis should be attempted. The x'easons were as foJlows: 

I ) It Wcis <*sHentlal that tlie profiles should be 
brough I to a reasnnab 1 e s tate before the 
operational service, bnl thereaftvOr ciiey sixonld 
be le Cl to opei-a to undJ s turbed imJ es^ tji<^re 
were comp(*Illng x^casons fox* modif i,ca tic n . 



::) Tiiii'» con hi tni\\ \>t* devoJ.ucI to iaj-/^o-.scalr 
uiocl i I'i (;;liJ on III i.lif. oxpeiiso of other snoro 
r/^Mc^ral ;inaly5>.i.« and It w£is considered inmox-tant 
to dLscovor ^jcn.jral orrocts rather than, tinker 
wit}) individual profiies. 

3) There was .some reasor. to Tear- that a "swings 
and j-oiindri;,.MM." erfect vas be^innin^; to show, 
wLtli modifications Dar^eJy changinfj: the balanca 
iK-twcen Kocall and Precision, but with no 
ov<* ra I I (T-a i n j n performance . 

•'0 The true worth of the data used Tor analysis 
was in que^iion. It was Telt paxM: LcularJ y 
that the list iiiarkin^vs did not prcclso'y and 
oonsLstontly indJcate t]:i- documents pertinent 
to I he ij^or>« interests. ConsiderabLo 
variations were known to occur be tween^no tirica lxo^» 
markings and list markjntjs. Until the uxc;ent or 
those varlaiions and the relevance ratin^^ oV 
iho Ltciiis marked could be assessed, further 
work basfid on this data was considered imwisc-. 

3) AH our proJ'ile compilation and modif Ica^ipji so 
far had bt-t n aimed at a f^eneral profile stand^rrd 
;7lvin^ a baJanco bexveen Precisici and Recall 
of app7*oxim;.tely 65 per cent Precision and 65 
por cent Rr-cm.l L. Even assuming that users, given 
t he Clio Lee, would have profo^rrof] rhis type of 
per forma ncf J thex^e was no reason to ^^uppose that 
ifi'-y* won !.d have calculated these Ti^-ux-e.^ the 
same basis. For ex^imple in detoriiiMiing PrecXi>ioji 
: wo to6k both relevance 1 and -riiievance- 2 documents 

notirii-d as a propoitton or tqtal nc^Mli cations . 
- ^ however Tor a user whosv p.o tifi oat Ions u.OJislst 

mainly of rr-levance 2 and ix-rplevont items, the 
f ac'ual perfurmance of the si^i-\'lco rlepi-nrj^ on tht- 

wo.'th of tin- relevance 2 items. 

It had aliv-ays been accepted that the typo up perl or:;:nnce 
required by users would differ f-^om individucil bo individual 
and that it was Important that profiles should attempt to 
satisfy Jr:dl-ldna:i r'-rjul rements t It would Imye been very 
usr^fni to have information about the; perfor-nance 

rerpiiix-iricits of InUivl ..at-' in the early stages of profile 
compilattnn l)u t th i > w.;-. r,ot possible since usr-3\s ruM^d 
e-vperionc- t>f an acmaj. SDI service belor-c- thev can iiu-dio 
meanxn^rful judg(Mii(Mi t . of tlie typ^ or perlormanGe they 
would wisii to liav(» 1 ro?u sucii a servif,'>. 



Prorn (|ucs tionna lr^- 

AJ'Lor two mrml.hs of w^-fMcIy service a questionnaire was 
scjnl. I.o iiscjrs l;o ascertain, amoiifi:' other thiii^js:- - 

a) The decree of satisfac tion or dissatislaetion 
5/it:]i Lho service. To a iar{jo extent this va- 
the main purpose or .the ques tiolmaire yiuce it 

' was iinportanl; to Icriow ±V any signiricant 
proportion of the users were sufficiently 
dissatisfierl that they wore liable to drop 
out. Tills also had the advantage that it 
LnriLcatcd rapidly, wltliout the need for tinie- 
consumin/; analysis, those users who were 
recelvinf^ unsatis fac tox-y service and whose 
profiler's nofded attention. 

b) T[io aspects of the service which appeared 

J east s.vtls factory. This was particularly 
valuable for users on the frin^o of the field 
who were dls^5a tisf led with Recall. It was 
possible to assess the extent to which this 
was due to hiljeaent limitations of the service, 
e.fj. limited subject coverage. As a result We 
were abie to explaii? the reasons for poor 
performance and suggest tliat users might wish to 
opt out where further participation seemed 
unJ ikely to be rewarding. 

c) A more precise idea of the iniportnnce >^of the 
J terns iiiarkefi "2" on the relevance returns. 

d) ^ Tht-'MuinbcT* of relevant articles thought to 

be published In li^e user's subject field each 
yeax*. 

• c) The type- of performance which the uscx^ would 
Like his profile to proAride. 

or the ii^f»rs wlio rtpli.e'^! .some 12 per cent cr*nss.dcrcd the 
service they had been receiving Poor or Very Poor and 
thi^'se profiles wre ^^ivon urgent attention to deten«ine 
wliat the fault^s w'-re and iiow they could be eliminated. 

Tiie questionnaire is shown Ln Appc^ndix 9A. 

At this stagG it was be ojfiing clcax* that the perfx>riiia:icr 
fi/»Mres ga^'^o only a r ouijli iiulj cation whethex- a profLJi? 
was operating to ti.c^ sa tisf ac- tion ol the user ;uid that 
ve:;y considerable ofM'ort was needed t:o monitor over 
500 pro riles and to analyse the reason for Jow performance 
Xr; particular ii was cJeeir that the samples of documents 
W.Ned for de tcrniliiln/: Recall performfince were far too small 
^' * effective judgcinien ts to bo made of the Kccaxj 
P'^rformance of indi.viduaL profxles and any atlei:;,/l to 
adju;,t individual profiles oC the basis of the figures was 
lu.Kol^j'^ to be dangerous. ' 



In /•..•no. ul U .i.s tr.H! thfif; data for asscssiDfe-- Precision 
porjnniit.iico is much moro roadLly obtained and reliable 
than j.hi.L lor Recall. More particularly, the user or an 
j.nforiii.> t i.on :.vsteiii is much more directly, aware of poor 
Precision than of poor Recall. From the practical' point 
of al ienii.txn{; to satisfy customers it is undoubtedly more 
rowaj.M'!ii.': to conc^.' trate on good Precision than on good 
He-call wliL^;- it ma> be difficult to assess in a way that 
is relevant fco ol'^ user's real needs. 

Given that the effort sp«:i!*- on improving Recall was not 
proving i)articularly fruitful ..nd also that limited time 
was available for profile analysis, nnd modification it 
was decided that greater empliasis should be pJaced on 
i.mprovlng Precision. From about week 030 in general 
efforts were made to reduce the amount of time spent 
m routine analysis and modify cution and to concentrate 
on proixles which vbv'-iousLy wex^e performing badlv or 
which were the subject of conmient by participants. 

In this coJu-l^«ci,i.on Li. shonld be pointed out that all 
comments by par ticip.ii.ts were scrutinised iimnediatel-.- on 
rccexpt and action taken to adjust the pr*)file as 
appropriate without delay. As a result the profiles of 
fhe most reactive pa 'ticiprin t^= received ver>- mucli more 
attention thaii the average. Thus some profiles were 
modified as majiy as fJrv'e or six times within the 12 months 
while otiiers remainorl urra4.^cred. 

Time spent -in "analysis and modiflT&aJiion 

The reasons for analysing the performance >f.4i profile and 
attempting to modify it were numerous: 



a) User diss''* '* ^^^' W* ^^i- ' 



The- siinpiesr. way Tor a .M%;er to drnw attention to 
• ursatisfactoi-y service was to put a ilcJv in'- 
he box at the b-ttom ri^cht hand corner oF the 
relevance assessmenf: return. Monitoriutj of o • I 
returns ensured that theso particular ororiles 
could be examined urgently. 

b) User comment 

Users were (-ncoura;jed to use the back of the 
relevance a-»sessment return for any comment 
whether in con jiuic tion with the <dl<>sa tisfaction 
box* or not, Ai^ain all returns were scrutinl>ced 
for comments and iimpedlate action takoiu Such 
cr>mments could take many fonns but normexj I y 
cxpj. ailed wliy particular items wex^e considered 
irrelevant, or defined tlio user<s interests morc^ 
closely, or asked for additional subjects to 
be included in the profile. 



(•-) P(^r-]-()riaancc the px-ori,Lo r^ppeared to the 

SIJT s<;o,('r t(i bo belov average. ^ 

Tlio Proc i >icin poj-ronnance .qi? each individual / 
pioriLc was bho i^osponsibili ty of a paj^ticular 
indoxer/ajialyst vho iiioji L Loa-^ed its poi-rorinaiice . 
Lii addition t\uy clerk i^osponsiblo Cor recording 
Uic rol(?vano(^ rotui^ns (Jrov llie attention o£ 
i.lio indcx(M7anaLyst to any prorile wliich 
«-jpj>oar<Ml to be perCornilTie badly. 

(j) (iiH^sU onua i. ro returns 

Tli(^ ionnaire sent to u«oi^s s min-dia tc^y after ' 

llic* start oP tlu* woekJy service war. intended, 
aii.on{5 othoj- tliin/-s. to discover which, proli I es 
wi re pcM-(oi.mLn(: uasa ti$rac torily • All.pronios o£ 
u.>iM\s who a-sessed the sor^v-ice Poor or Verv loor 
were c^xaiiij lufci ure^ent J y and Later aLL those' of 
ns(?rs wiio assessed the .s'T'xvice as Fair* 

Tlio anaiy.si^ and J'lod 5 Pica tion done to the profiles is • 
snmiiiarix'»d In AppendJX OB. 

in coliunn t j-^ /v'^^en tJie sei^ial number of the profile ^i.J 
Ln coJninr. the reason for the analysis- Mosi|^of the 
reasons are self-expi anatory except' perhaps ui)utine' 
whi ch means tlia f the analysis was carried out Tis a iref-ular 
part oC the business of performcuice monitoring. 

All .profiles were, of course, monitored in thi:> i-avT *'ohlv 
tiio?^«* whose Recan or Pi^ecision .performance was oonsldored 
imsatisfactory oi^ tlie, basis of the fibres were anaiysed- 

explains why some profiles are shown ai> having had no 
.-uialysLs or modificarion done to them- 

CoIlL-ui 3 of Appendix OB ^rivcs the week in whicJ the' 
modification took eriect and coliunn h the purpie of tlie * 
modification, R meaning the modification was a*ied mainly 
at impx-oved Recall, P at improved Precxsion an?R/B a 
i-^iuv.a improvement i.> increase both Recall and Precision- 
CoMjmn i> (-xves tlie Lime taken for the combined jfork of 
anal ysis and mod i f Lea tions - In column 6 is the hser^ 

assessment of fciic- sf^rvice he was receivinfC approximate L v 
?wc months after tin* start of the weekly servicfland a-ain 
some twelv.j months .later after atl .^re anal y.si.s *nd 
modi^fication of iiJs profile- TJie abbreviation:, used are 
VP ^ Very Poor, P - Poor, F - Fair, G - Good and VG - Verv 
Good. 
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Chapter 10 



JNTBRACTION WliTJl 1>ART:i: C.I PANTS 



In any i iiV<*3 t.i /^a Lion which Involves the voJ.uii tary parLicLpa- 
tion oJ: o\it;.*5i(h* peop I o., tlie iiiodc or Xii terac txoii with tiioiii Is 
.1 mporLari I C i.von Uw* pro'lon^;ocl x"C{;ii I ar n^sl.s i:ancc' neodod 
rroni nartic i pan ts in the SDX Ijives ti^ation , the os tablisiuiicnt 
and con L Lriuatiori of (vood relations with thciii was vital to 
the project. 

In tlio d:i.s«-us.si on oi* the sample of people used Tor the 
Investigation, mention has been made of the problems 
assocjuatod vi rh drawinfj a lar^^o random sample and the 
dxffl'tulty of ensuring; that the finuX sumpLc-, allowin^^ for 
expected wasua/^e, Wiuld be lar^vc enough for all requirements. 
In these c Lrcums tanc<*s It .vas essential to ensure tliat ill- 
considered action on th(^ pan of the inves t j /^ator:5 did not 
precipl tfitc loss of partic LpanL^. , 

Vhen^it i:? consldcrt c! liow' many i.ndividucvl actions were 
required from j)art i pants cind the fact thv.l Vot tlie first 
12 months or so man> of tliem received very little useful 
service, the temptaiion to drc-p out emd thus save time for 
;^ther purposes must have been very strcaf^. 

The fact that .such a Jar^^e pr-oportion of partLc pants 
voncinucd to the end can be attributed to various causes, 
but undoubtedly the r^Miexx):^! ty of tho partioIpanli> In 
responding, to an anpi-al for assistance Ln an Lnves tl^irat Lon 
must be ai(.on{7 the chcf^f of Lheiii. At tlie saiiu* t "uiu* ti)e staff 
of the Jnvcstxcation wero wei ! aware of tlie need to retain 
the {goodwill of participants and to simplify and minimize 
tlie vork required ol" them. 
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In particular vnry r.roat ofrort woiit into the drafting; of 
N»ttor\s to finsuro tlu-y wore c.l.»Dar and concise. As Tar as 
possLhlo propri.nted rox^ms were supplied for replies and 
in all cases whor-o a reply was required, a business-reply 
envelope was enclosiKl • 

The art of p^t^rsuadiii^v participants to devote time to 
answerinf^ yet another s,et of questions » while retaining some 
fi:oodwill for the next time, is not an easy one but the problei 
is common to most investigations, many of wjiich have little 
else in common. Tliis aspect oC aii investigation is often 
ignored in the f ina 1. report, with the result that the art 
is not furthered and each investXfjator sets about the task 
from scratcl^ - 

In the hope that they may be of use to others the letters 
sent to participants during the investigation are given in 
the Appendices. ^.W^^ ^ very great deal of the Investigation 
directly involved gllrticipants , the Appendices give a gbi'ierai 
synopsis of the course of the .ork. 

Appendix Letter a0;ompanying the questionnaire. Survey of 
10A(i) Information Use by Research Workers. Tlie letter 
was sent to sevcial thousand randomly selected 
electronics research workers. Those who replied 
became eligible for further selection to form 
the User and Control Groups for the SDI Investi- 
gation. 



(2) 



Survey c^t^ Information Use ques tiojuiiire , 



(3)^ Follow-uj) letter 

(^f) Invitation to research workers to take part In 
the SDI Investigation 

(5) Explar^a^B^pn of the SDI Investigation and tlie 
work itfvqUved for participants 

(6) Statemdnt of Information Requirements form 
( 7 ) Sample s liatemen t 

(8) Invitation to librarians and information 
officers to act as Project Associates in tlie 
Inve s t i ga t i on 

(9) Role^jjl^ the Project Associate' 

(10)^ Notiis for the Project Associate on the selection 
of User and Control groups 

{!))* Description of the SDI Investigation 



(I2)jr Follow uj) to loiter of Invitation to research 
^ workers -» 

(I'j) LeLLcr scM^k Lri/; c J a ri fica oion oT information 
requiremcui Ls (sent to approxlnm to Uy one 
quarter oJ' participants in connection with 
investigation of profil.^ compilation method?) 

(l^) Reply form for above 

(15) Letters a.sking participants to mark list of 
(J6) documents for initial testing of draft profiles. 
in) (Tlxe exact form of the letter received by each 

])9,rticbaut depended on how early he joined the 

Jnvestiga tion) 

(18) ISTotes on Tnitial iV.Sijessment of Relevance (The 

(19) form of the notes differs depending on the size 
of the test collec'tion of documents at the time) 

(20) Reply form to accompany Initial Assessment of 
Relevance? 

'(21) ' Request for relevance assessment of second set 
of docunionts 

(22) Reply fo rm for above 

(23) Letter to participants asking them to, check the 
draft profiles (sent to sample of approximately 
one quarter of participants. See note on 
.Appendix 13 above) 

(2^)^ Reply form for above 

(25) Letter arranging Visit to discu.:s profile (sent 
to approximately one quarter of participants 
(-se^.noto to Appendix 12 above) 

(26) Reply To rm for above 

(27) Letter confirming visi^'t 

(28) Letter seeking relevance assessments of a further 
sample of documents 

(29) Follow lip letter chasing non-return of previous 
relevaiiCG ci^isessment 
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i'iO) Tliank-yoii letter ;icknowledgin{j return of 



rol.ovuncj> assessments 



(:J3) Invit.'it ion to Group participstnts 

(3^) Explanatory notes on Group participation 

(35) Informatton Requirements for Group and 
questionnaire on its members 

( 36) . Acknow'J (*(|/»vMncut of Tnf^ormation Requirements 
Trom Group participarits 

(3.7) Follow-u[j letter chasing no return of 
relevance^ assessment 

(38) FoLlow-iip lobtex chasing non- reply to draft 
prof i,l e . • 

(39) FolJow-up Letter chasing non-repjy to queries 
cn px'of i, I 

(^*0) Letter to Project Associates informing them of 
start of experimental service and asking 
wlietlier notifications should be sent via them 

(^1) Reply fovin for above- 

^Gttea- acconipan3-ing first experimental notifications 

(43) (later vcj;siori wa>> sent to participants receiving 

' first no t i.f ications ^ome way through the experimental 

service period) 

(44) Explanation of relevance assessment markings for 

(45) SDT noti rications (second version for participants 
making returns via Project Associates) 

(46) . Letter to Project Associates, accomp£mying first 

no tificc^ tions 

* 

(47) Letter to Project Associate, accompanying second 
batch of experimental notifications 

(48) Request for relevance assessment of sample List 
of documents included in first experimontal 
service runs 

Letter to Project Associates explaining relevance 
assessmen ts 



Letter asking participants for relevance assessments 
of sample lists of documents input to experimental 
service runs • '""^ 



Lv-J Letter ;i.skj.n/.7 Vor further rolovanco assessments 
{')')) of sample? lists of documents 

('I^O SDl Newsletter intended to inrorm participants 
or the i)rot^are.ss or the Tnvest.L^^ation . For 
various reasons no later issues appeared 

(55) Letter seoking clarification or participants' 
reasons for markin^j particular dociiments as 
reJ.evant 

(56) Letter seeking relevance assessment of additional 
documents selected rrom test matching run by 

o vi s e d j ) r o r i 1 o 

(57) Fol.low-up letter ciiasing non-return of relevance 
assessments on document l^sts 

(58) Coverxn/v note to Project Associate accompanying 
'cliaser^ letters 

(59) Letter to participants who had failed to send 
in relevunce asscissments for experimental SDI 
jio tifica tions 

(60) Reply f orm ror above 

(61) Letter asking Project Associates whether weekly 
notifications should *be routed via them jbo 
participants 

(62) RepJy form fo3^* above 

(63) * ■ Note ompanyijig first no tif J cations on new 

preprinted two-part computer stationery 
» • * 

(6^t) Note accompanying supply of business V^ply 
envelop(^s to Project Associates 

(65J Note roqiicsting certain participants to return 
relevance assessments direct rather than, as 
before, via Project Associates 

(66) ExpLanatory letter accompanying first weekly 
operatioiial service notifications 

(67) Letter asking participants for relevance 
assessments of sample list of documents 

(68) Letter seeking furtlier relevance assessm(Mits 

(69) Letter to Group coordinators .Peeking relevance 
assessments of sampJe lists of documents 
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(7^0 I.fM ! ci- ;icc()i!ipnny i iir; SDL Sotvico (idcs t i omifi Lrc 

(7^) HDI S(-rvi<^i^ Qii(\s t i (MiiiiJ i i n (.(iiidcul to cllscovei" 
V(^;irl\(3i\ par CJ (• i.pan '..s to LFk; I*L7,\st two 
inon iJis <) weekly Wloiy » 

(7^) f.cttor to par tj.cJ pan ts wlio had not sen^t in 

rnilnvanro x^c turns ror sojno considerable time 
and whose con^.lnued [)ar ticipation was in doub, 

(73) Hoply r orm for above 

(7^1) Ronii.ndcj- to pai^ticipan ts who had Tiot sent in 
no turns for a inoniJi or so 

(75) Letter roplyin^j to particular comments on the 
SDI ServLce Quest i onjiaire 

(76) Note e^pl.ainlnfj prc^sence of older inafcorial 
in notifications 

(77) Letters seeking data for study on variations 

(78) in the marklnff of Lists of sample documents 

(79) Letter accompanyiuc" sample ,1ist of documents * 
for re.lf^vancc assessment 

(80) Notice of Ler*inination of service to participants 
who faJ led over a long* period to send in 

re 1 e van c o returns 

(81) Coverin£>; letter for fia:^st notifications despatched 
to a e:i*^»up of participants joinin^^- the investigation 
at a La i,er stage 

(82) Use of autliors as search terms - covering letter 
(H3) Reply form for above 

(84) Particii larly valuable periodical articles. 
Accompanying: letter for .^tudy aimed at discover;! ng 
the per J rmance; of the system for particularly 

• valuable articles 

(85) Reply i'orm for above 

(86) Not<»s for study on novelty of in foarjiia tion 
supplied by the SDI Service 

(87) Note for study on use made of no ti:f: cations 

-(88) Letter accompanying sample lisi' of docunuMjts 
for re 1 ( » va.nc e assessment 



(^7) rai' stijdy an ijs<* uuulo oV no I i (McriLloiis 

(8H ) \,(iiit^r ;MM:oiri|)any i n/v sain])l.o J .ist oV ciocumen (;s 
Tpr r<^ 1 !• vanco asses .sment 

(89) Note ox|> I ainiruj: temporary change of computer 
stataojiory owin(,»: to transfer of computer 
service 

(90) Lotter asking for relevance assessment of 
sam]>lo List of documents 

(91) Altornati.ve form of SDI notif ica t Lons • Letter 
aski.n{i- for users' views on the continuous 
compute J- printout as compared witli tlio normal 
cards 

(9^-) Not(* accompany iii^v card notifications delayed 
by the above s^;udy 

(93) Letter seekin^^ assessment of articles selected 
on the !>;xsls oX* author nanion 

!*(9^) Relevance assessment of fu*l article for use 
in comparison with assossipent; based on card 
no tif ica tion details 

(95) Covering:: loiter for second »Surv'-y of Information 
Use by Research Workertj^ quos tioniiaire despatched 
towards the end of the SDI Investigation. 

iOvr) 'Survey of Information Use* questionnaire. 

(97/) Letter Lnforminfi' Pjroject Associates of the 

conversion of the Tree SDI Service to a cost- 
recovery service 

(98) Letter to participants as above 



As can bo seon^ approx Lmately one Jmndred standard letters 
and forms had to be compiled and despatched to each of tlie 
600 participants. This in itself took a (.^reat deal of effort 
but dt does not inoUide the many individual letters that 
had lo be compos^^d and sent to participants to cover 
pa.rticular probleins. 

One of the chief needs lor individual betters arose in 
<"onnection vrith the comments rbce.iv^'d on tlio weekly 
r<-levanco return::.. 'IliJ s problem was met by ha.vi n/^ a standard 
reply letter J caving space ioi handwritten' indiv LduaJ oonmu^nts 
\)y the sat si;aff. 



I (. cannot bo TJ* i c i (mi I: I y (Muphas iscd that an SDl Service 
is not a ^xn^lo. onl. i(y bul; niorr* I.y the sum oP all the 
indiv.iOual services provided to the individual custoinei^s. 
Interaction vith the customers en ]:)loc is onJ.y possible 
to a limited extent and one must always be prepared to 
{^ive special troafcmont to smaller ijroups or individuals, 
though this is Inlierontly expensive in time and money. 

Within thp limits .oV thg time and stafM* avallalDle it is 
oon.sich'iN'd i.hat the* i.n tei'ac tion wi tli participants was 
successful and that this Is shown by the coopc* rat i on rocclvod 
throu^jliont the 1 nves I i {;a t i.on and I)y the i-clat.ivoly low drop- 
out ra tr» ol' par t.i.c Lpan ts * 

Th<* oJv*^im(Mits that ai thou/^ht to have contributed to success 
arc : 

1) C'.onsidorablo attention to the coinpusLtion of letters 

2) Individual addressing of communications 

3) Personal si(i*n Ln^^ oT most lotters 

^} ) Parovision ol reply forius in almost all oases to^jcther 
with ropJy-paici onvolopos 

5) *Cliaser« letters whore practicable enclosed a second 
copy of the reply roi^ni to avoid recipient liavin^j to 
f*ind the or.i/yina1 

(S) Kmmediato sciutiny by the Xnvesti{,jation Mana/^er ajid 
acknovledf;omont oX' all comments and complaints 

It is IJ.kely, on the other hand, that many aspects our 
interacllon could have been improved. One that we are aware 
or is the extent to which participants wore kept inTormed 
of the jfjcneral p^.*oc:r(^ss and results oP the Invesi>Ic^^tion 
in which they were njiturally interested. An attempt was 
made to achieve this by means oP ah occassional Newsletter, 
but this fell victim to the need to spend available time 
on more iimnediAte.]y pressing? niattors. 

Taking aXl thin{5s into con-:ideration it considered that 
the interaction witli participants v/as satisTac toriJ y done 
and could only have been siffniricantly improved by havin/; 
a larger r umber of s taf /; en^ja^sed on the invos ti^ya t ton or by 
a small number oT partlcipaji ts . 
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Chapter 1 1 



USER COMMICNTS ON THE Sli^RVICE 



Towards t:li(* am] oV tin- SDI I iivo.^ L i.^ali on afLor par i- i c i pafits 
ha(i ho«»n n^iu-ivlur; siM^vic«> for o oj- 12 iiiontlis a PLriaJ 
quesl.Lonna i rr; was s«mii to tlieiii l.o discover LliiiLr views on 
the service tlioy had each received. 

In addition to specific qm^stions on various aspects of 
the service, participants were asked to ^rlve any ^^^eneral 
coimiiejits they wished Lo make on the sc^rvice or tho 
Inves t l/^at ion in /^ein.Tal. 

A considerable number of people took advant?? ^e oV the 
oportunity to ampLlPy replies to particular questions 
and to comment on tho service a.s a whole. 

Owln^; to the varied nature of the comments, no attempt 
has been made to sort and classify them. They are /-iven 
^ below, arran£^ed undei- the serial number of the participants. 

Since the questionnaire appendix was sent immediately after par- 
ticipants had been informed that the free SDE Service was 
being converted to a cost-recovery service, the comments 
naturally In many cases mention cost factors associated 
with a subscription to the service. 

007 

The most useful improvement would be to liave the profile 
updated, say, every four months or so. 



on 



r think It would be useful If "Inhibitor keywords' 
could be ImpJemcMited In the schc^me nioro quickly, 
e./;. for a lonr, time I recoivc^d notification of 
superco nduct In/', fi.hiis i,n Conner t i on with my work 
(which are to me quite un In tei^es t In^^ ) : it would 
have been nice to exclude this cate/^ory. 

r think any comments, object lon.s etc. ar<* Irrelevant 
because In spite of any ci Isadvan ta^jes this type of 
system must Inevi ta})ly replace the old technique of 
tryln/c to scan all journals efc. personally since It 
has become an almost lmpossll)lf? task from tho I iiie 
requ Ired. 
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Tt does not absolve one from reading all of the journals 
that normally publish papers in one's own fi.eld. One has 
to buy some (as many as possi.ble) of these anyway for one's 
own reference library. References outside tlUs f J -sld by 
SDI were" relatively few. ' . 

022 

The sorvlc(! was most sa t i s far toj-y on the wliol.;. My pi-ofilc 
was only modified when /;enora.l modification Lists wcro issniMl, 
1 would have proforrr-d this to be moro f i-i-.jikmi t in re trospi-c i. 
althou,;}! L did not really notice chan/ves al. l-he time. My 
cessation of use of tli i s service is duo more to mv research 
boin/r ended rather than a lank of (h^sire to continue with 
the service, althon/vh I doubt if "Which" Av-ould term i( 
"except ionaLiy r.ooci value foj- money" at £^1*5 p. 



>a , 



12 



Some of the articles 1 have been advised about have been 

useful to members of my ijroup who mi.v^ht not otherwise 

have heard of them. In this respect the Impact of the 
service has been widened, 

024 

C am sorry that I cannot spare time to /rive considered 
answei\s to all the questions on this form. Ihit hero Is 
a general comment;. My r.oneral standpoint is that the 
SDE system seems to be a r.ood ulea. ; ts plan is (pi i to 
reasonable en and the results have come up to expc. c ta t i ons . 
BUT the question, of course. Is "Docs it fill a real 
need and who has that need?" The positive ansAv-er A^'ill 
come from ouhcrs no doubt. For those of us iji the for- 
tunate position of havln^^ acess to a University Library 
and of bein/i a member of a research team enf.at^od on 
rather speci.alized work, the r.ood does not soeiil to be 
fjreat. The point Is that tlie number ol' Journals iii which 
relevant articles appear is small and we have access to 
thorn all anyway. To be honest, SDI has not drawn my 
attention to any article which F would not have picked 
up in the ordinary way. So I must -Cf vro tfn 1 1 y ropoj-t 
that, al t:hou,jh the system is sure lo be attractive to 
many, it would certainly be an nnnecessa -y luxury as 
far as I am Cf)nc(.'j'ned . 

029 

The service allows me to see relevant I i te ij, tiii-e i<r|,en 
often i am really too busy to ex[)eet to know of any. 



ERJC 



m 

The SDI Service i.s too liiiiLfcod. My usual requirement 
(and L'ln sure fchi.s will appjy to many other people) is 
that, tVLvon a new topic or some particular aspect of a 
topxc, to obtain all relevant articles published to date 
on thai, subject. Tlio present method .is to scan through 
the abstracts journaJs, and although this does consume 
a Little time it Is satisfactory. it ought to be possible 
to use the computer to do such a search in a similar 
way to that which you presently use, but, of course, on 
a larger scale. However, L do appreciate that a vast 
amount of literature has been published In the last 3O- 
40 years; but even so I! think that an SDI service offer- 
ing a literature list of relevant articles published 
over a limited number of years would be more valuable 
than the service offcired at present. 

036 

With re(.7ard to question eight. It would have been more 
satisfactory In the <:arly stages to have better checks 
on whether relevant articles wore being missed. 

X am not joining SDI system, not because E don't think 
it a good idea but bocause In the university envlrormient 
we already have good library facilities. 1 support the 
idea for industrial use because of difficulty In scanning 
many periodicals and because of time factor. 
Sup^gestion; overprint back of card so that an integrated 
filing system can br> built up. Some work study -could be 
done on this aspect. 



If the services included those items listed under question 
7 then to an lisdividual fche cost of the service T would 
have expected to be equivalent to the fees charged by 
professional institutions for membership (£5-£l4) a year, 
with an entrance fee to cover initial profile data. * 



J can only repeat that I feel that an abstract is essential, 
1 would Ixke to thank you for letting me participate in 
this experiment. L have found it most illuminating. 

In general the service was satisfactory. Peripheral 



11-3 



arclcJos tended to missed but this was probable due 
to the difficulty of Including' all aspects of one's 
intert^sts within the prof LlC« 

06^5 

We should have a statement of our profiles every 3-4 
months and form for amend i.nr; it - many requirements chan^ve 
and we don't really know exactly what the proPJle consists 
of al tor;e ther • 

We should have a iisi in^-, by sub.iect headings , of the 
authoi: and ti.tle oril\ of those Items we have marked "1" 
and "2" every three-lour months, and we could qui.ckly and 
easily scan and review the recently notifi.ed articles of 
interest in a particular subject - cards r;ct bulky. 

067 

Under (n) what do university research groups do? Either 
the sej/vice is very (ixpensi.ve or one writes a ^^roup 
profile which seems f.o me to stretch the rules T 

More selectivity seems to me essential - otherwise one 
^ets snowed under wi th trivia. 

022 

1 believe that a sei-vLce of this kind is only justifiable 
f or a research worker with enou^jh time and incentive to 
follow up the notifications. The profile should be highly 
specific. Some way mus t be found of keeping the cost in 
proportion to an individual's work to liis department. Once 
SDI is well establisiied I i.magine r;roup subscriptions 
could be instituted - eg an Institute pays £5 pa Tor a 
number of individual workers (this number fluctuating in 
any one year). 

For my own type of job (which J von' t attempt to describe) 
one is liable to be confronted _wi th a technical qestioii 
suddenly and therefore to want a survey. t should like 
to be able to ring up someone and say: 

1 Can i now obtain a iow noise MOS transistor cheaply 
in this country and if so from wliom? 

2 I should like to know of all theoretical papers on 
"Cockfof t-Walton" typo cascade voltage multipliers. 

078 

The service improved appreciable over the ye-.rs:- 
Xnitially it notified nie of un i niores t Inr; a:»*Iic3es. 
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Finally i I. uol\r\(jt\ mc* whcui a Joiirnai oV Kril.(-rcsi lo mc 
had bcMiji published* 

In its present stato (:}ie servlee provides valuable back- 
up, but I would not only rely on it to keep me in touch 
wi. th the literature, 

T somoti.mcs wonder wlicther the Peedback mechanisms, 
between user's i^ntcrcsts and his profile i.n the machine, 
is stable. Tn my case I. think Lt lias been but T can 
unaylnc the possibility that tlio profile mi^-ht cncourae;e 
a user to spread his interests more and more wLdcly/thin. 

092 

The service In r;chox-al was very useful. TJic main problem 
Ls xri the user dofinlnij hi.s profile. There are always 
papers whose ti.tlo content cannot be prodi.cted and may 
not fit Into the profile but bo of interest. For this 
reason it is still necessary to occasionally scan tlie 
publications personally to onsui-e that nothJn^^^ of interest 
escaped the profile. 

T found the covcra/^(i {^ood when it comes to deal with a 
fai.rly wide and well defined subject (superconductivity). 
It was inadequate wlion dealin/^ with a fraction of a Ticld 
(thin films) althour;!i in fairness I doubt whether I defined 
my fractional int'ercst well onou/vh.. This was due both 
to difficulty of definition and to the fact fhat my interest 
shifted with time. The last coimnent is an important 
reason why f .prefer to use CPP: the scope of my interest 
can bo varied w i tli time and 1 know what I have left out 
at any ^-ivon time. The other reason is, of course, cost. 

Et would be very useful to be able to abtain notifieation 
of all articles published over a certain period of time, 
the past two years for example, by a. specified autlior or 
groups of authors, or from a particular or/jran isation, or 
on a particular narxowly defined subject field, etc* (l 
appreciate, of course, that an extra charge would have 
to be made for such a service). 

098 

My particular inlorc^st is peripheral to normal require- 
ments from SDI', but It is certaijiiy useful, tlicroforc my 
comments arc perhaps abjiormal. Main problem is tliat of^ 
inissin/v substantial number of articles. Has my letter to 
you of 21st January, lyyO hclpecJ in formulating^ uioro satis- 
factory pro r 1 1 o . 
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I (.hiiik it. imp«,i(.a,ii for (isors to check ovor Lhoir profile 
ror.ularly - say .:vory Uiroo irioriUis, to combat the problem 
of iinssod arti.cl-s. Tlio most important factor - to me any- 
way - of such a service is to be certain as possible that 
all relevant articles in the journals scanned are notified. 
If this problem were solved the service would be invaluable . 

021 

In reply to Question 11 my exact position is this:- 

1 T would have likfsd to subscribe 

2 Funds arc vai^y iuniipd at the moment 

3 Honco I woul<I pre Per £35 worth of equipment for research 
than the SDI service. 

I would be prepared to sacrifice up to £10 worth of 
equipment for tlia service at present. 

5 in normal financial circumstances at the college I 
would have subscribed in view of the time saved. . 

6 1: can fjet the inf ormation 1 need either from the college 
iLbrary or Patents' office library and although this 
takes longer it does not cut into the research .grant. 

I 

7 A group subscription was considered out of the question 
as the Interests of my colleat^ues are too d.i versified. 

101 

As mentioned in answers to question 2, I have received 
rather .a lot of notifications of papers on semic6nductor 
physics - effects of doping, group III_V material etc. - 
whereas my interests are only on the application of devices 
m circuits and circuitry. 



Too many sources of papers in my research field were not 
scanned. 



I feel the service is uscpule for fringe topics but the 
main articles oh co.smls ray air showers do appear in the 
Proceedings of tfie IMennial fUPAP conferences which 1 
normally purchaso. 

As I commented carli<«r, L am i.ntorested In large detectors 
of radiations. The profile <.herefore included radi.atlon 
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doJ.ocf.ora. ant5 mos I. of l.ho cards sont to itio on this heading 
worn oV li.tl.Io i n (.r; i-.;s (, . 



109 



1 T think the customer should be invited to update his 
profile more frequently. An explanation of how each 
card in any one particular week came to be chosen would 
perhaps allow tho customer to eliminate unnecessary 
cards by alterin^j his profile. 

2 The information r;iven on the card should allow the 
customer to write away immediately for a reprint. 

110 

While I received no t i.f ications of most of the relevant 
information there was a preponderance of irrelevant liter- 
ature. I do not honestly consider that the service I 
received - useful as it may have been - was worth the amount 
you propose charging;. Inclusion of foreign language reviews 
xs obviously of paramount importance: anyone can scan an 
English language paper, but how many can scan, as easily, 
a forexgn language paper or report? If anything, I would 
say that this sorvico would be more valuable than the 
English-language service given at present. - 

111 

I have found the service of groat value particularly • for 
fringe journals that are not readily available. For* example 
our departmental library keeps Physics Review, Jo'rnal of 
Physics, A B C D E, etc., so I always see these quickly, 
but I have to go a cons Iderable way to see Australian 
Journal of Physics, etc., CPP is getting very late now, I 
regret to say. If they could catch up - that would be 
the answer. ' 

118 

Generally satis factoi-y , but too many articles of no interest. 
126 

See oarl'-er - it would be extremely advantageous to have 
the address to whicli to write for a reprint on tho card - 
as is done in CPE and CPP. 

It appears to mo thai If one's Interests are quittj specific 
(e.g. piezoelectric phenomena) then not only Is It easy to 
define one's profile - but all things follow from this. 
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One then obCai.ns a sorvLce thai is as g'cod, as the number 
of period Lcals surv<»\ ed« Rut clearly it is insuff Lciant 
to define one' inte^-csts in terms of words like "control" 
"network theory" and "stability of nonlinear systems". 

I wonder, too, how receiving: a relatively large number of 
"minor interest" reJ*(irences modified ones future receipts. 
This could be a divt^a-^gent process rather than a convergent 
one. 

For myseir my interests are fairly wide and E find that a 
good periodical lib^'ary visited regularly is preferably 
my most efficient pt^rsonal system. 

1 

ni 

I am sorry indeed to see tlie end of my SDI cards but i do 
feel strongly that ilie price either to me or my employer 
would be too high sj uce this amount can be spent better 
on buying the journeM.s wliich are most likely to contain 
the important information. 

137 

The service is useful to me but as most of the information 
I need appears in tliree journals only, it is no great 
handicap for me to scan those issue by issue. 

On the other hand if would be of real value if foreign - 
language Journals wf re searched and if translated ab- 
stracts were easily available (of useful articles). 

142 

Our difficulty in sid)scriblng to a data sorting system 
was outlined in the answer to question 11. The main 
objection is due to Lhe ease w:i th which we can get information 
from the university library and the cost of the SDI 
Service c£;.rmot be justified. 

One difficulty with any da(;a sorting system is the level 
of confidence it' inspires In its users. The ideal system 
would miss no injporiant (relevant) articles and could 
thus be relied upon lo reduce to zero the work required 
in looking throu/^h J Ists oi references of cux^rent period- 
icals • Unless a vei v high level of confidence can bo 
ensured the SDI serA Ice has little advantage since l;he 
scanning of lists oi references is still required. 

A facility for looking up past ref'erences at the start 
of a new research pi oject would bo very usedil. iCvon if 
this service provided a list of general refiiPonces in the 
area of Interest it would be usef\il. 
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^^'inaily t should iiki> to thank you for allowing me to 
participate in the SI)X Inves ti{jation and I hope we have 
been oT assistance to ycu. 

I have found it use Till and enjoyed seeing how you have 
built up this service • 

I don't think 11 (a) and (b) ax^e quite mutually exclusive 

1 I would have liked to subscribe 

2 The service had a high academic value to me, but 

3 tho cost was nevertheless beyond my personal financial 
resources • 

I think this will bo the case with most university person- 
nel. The noir.inal co.st to which most of us would agree 
would probably be uroconomic from your point of view un- 
less it resu..ted in a tremendous voltsyneo In the absence 
of the service I shaLl return to searching the very adequat 
literature in the university which is probably inefficient 
(from the university's point ol' view) but no cost to me. 

159 

The service has been helpful, [but not essential. For a 
member of staff in a University Department it is difficult 
to see how funds could be made available for this service 
on a personal basis. 

161 

My big problem is f uiding time for reading. I use your 
cards for two purposes :- 

1 to find articles for iinmediate reading. There are, 
of course, far too many, so I read very few. 

2 To build up an index, filed under subject, covering 
my general field. In the past I have done this job 
manually, based on my own reading - your cards speed 
it up and improve the cover greatly. 

For both these applications, short summaries of the papers 
would help enormously. Titles are often too vague, and the 
keywords don't help all that much. 

Recently I have had u lot of cards about nuclear radiation, 
its detection and its effect on transistors. It would be • 
helpful to have negative choices available e.g. nothing 
"concerned with nuci(iar radiation", " b^^^ Dr. Whitfield" 
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or whatever. Perhaps you do this already. 

Cost of £35 is definitely too high for large-scale use 
of service. 

Jil 

1 was disappointed in the apparent inability to modify 
my profile so as to reject references containing certain 
topics, despite several requests to do soi Far too many 
irrelevant articles were picked up compared to relevant 
ones and relevant ones missed (the approximate ratio 1 

2 X was 29^, 3l9i hoi). Also, many titles appeared in 
the current awareness publications quite some time before 
notification by SDI. 

On the pro side the cards, size and layout, are quite 
satisfactory, and inPLnitely better than ordinary print- 
out. 



For the future, IMii .sure some selection service along 
the lines of the SDI must come, but it must be made more 
reliable, both for pLckLng up relevant articlc/s and 
rejecting irrelevant ones, but I»m sure that this is your 
aim also. Perhaps a simpler form of profile requirement 
might help, such as Just a list of subject headings, in- 
cluding those which would make an otherwise relevant 
article irrelevant. 

168 

Too few journals were; covered by the service. I found 

that most of the articles referred to by the SDI Service 

were in journals vhi ch I read anyway. 

m 

I was working in the field of non-linear optics at the 
commencement of the Investigation but shifted the emphasis 
of my work into optical waveguides shortly afterwards. 
The service took somci time to update my new interest and 
some confusion resulted, which was eventually cleared. 

I found the service quite useful although a large propor- 
tion of the notifications were not directly relevant - 
but in that particular field there seemed to bo com- 
paratively few publications at the time in question which 
were of great signifM cance. 

However, the quality of the service during that time 
did improve confirming the usefulness of weekly returns 
and periodical .article lists. 
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My Lnterost: in non- 1. Inoar optics ended in September 
and r now work In tin- totally unrelated field of real- 
( ime compu tin/5. TJiou/jh I have requested that my pro- 
file be updated this has not been done. I appreciate 
that the period at wliich L changed was near the end of 
your investigation poriod and this could explain the 
inabilUy of the sei^vrLc.e to follow rapidly my change 
of interest. 

211 

Before asking university teachers to pay for tlie service 
you should have negotiated with bodies providing research 
contracts - MinTech, SRC etc. We are engated in frus- 
trating correspondence and lobby In/i with these bodies ' 
about Information service costs, and heaven knows when 
they will decide anything. Can you contact them? Uni- 
versities will not consider paying until contractors 
decide, I hope you won't take me to court 1 

178 

Would find service more helpful in assessing .i^elevance 
of articles if an abstract was also printed on the card. 
E feel that the Inclusion of US government R/d reports 
etc., is essential. 

Conference proceediu/js should also be covered since in 
general these repre<?(jnt the most up-to-date research 
Information. 

m. 

r do not think that, ;jiven the available funds, I would 
continue vith this service. 

My need is very much in the chemical physics field (not 
chemistry) and you do not cover this area. However, as 
you see, I have enjoyed the SDI. service. 

180 

The main fault seems to be the need for very carefui 
statements of the profile. For example, some articles 
of interest concemod with stabilization have been missed 
because this work di.dnU appear in the title; the 
reverse fault occurs with far-infra-red materials, where 
t ofter (^et articles of little or no interes( because 
it seems impossible to exclude them without missin;; some 
of interest. This Cnult is, of course, less serious. 
But T have found tho service useful, nevertheless. 
Sorry about my slowness in completing the form. 



181 

The (jradual improveimmt Ln the service, especially within 

the last six months or so, indicates that a muclx hl(ijher 
standard is a possib i.lity« L wish you further successes 
in your work* 



282 

The only conmient T vish to make, is, on your cards would 
you place the author '5 name at the. top of the card. This 
would make it easier for filln;^ 

Authors' addresses oni^Yi ^ to be provided on cards to enable 
direct application for reprints to be made* The inves- 
tifjatiou and service provided to be much more useful in 
general (:han the Lovry-Cockroft abstracting system used 
hitherto* 

.181 



II has ucon a pleasure to par t ic i pate .in IJie birtli of 
this new information service and 1 have derived, 1 am 
sure, more benofit I'rom It than i: have given you in return* 

One point is not cl(»;ir x'cgarding -Uie group usei' service 
and how It could be charged* You may know that one of 
my r \leaguos, Dr* V.J* Phillips has also taken part in 
yoi trials though iiis profile is completely different 
from iny own* Is it possible Tor our two profiles to be 
combined under the group scheme and thereby save us money? 
As the profiles are so different it would bo easy for us 
to sort out the cards when they come, as this we would 
lose Ij^-Me time. If this procedure woulcJ bo admissable 
then 1 think we would both reconsider our decision to opt 
out of the scheme* 



192 

1 have the impression that t}io "recall rate" (loss than 
20 per cent category I) :in my case was due to my odd 
profile* r think you did well to achieve this. 

I2it 

Taken overall, from my point of view,, the service has 
extremely useful* 



My one adverse comm^Mit re section 2 is that my work changes 
rapidly, virtually from month to month* To change the 
profile at this rate would be unsatisfactory; it follows 
that items relevant to the current job were not always 
available at the time required* As an overall method of 
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koopLiu; touch with niy subject then I am most satisfied, 
and SD.r on this basis is an ideal intrunient. 



One further comment of a more (jeneral nature, even with 
SDI there is still a lot of literature and still consider- 
able reading time has to be found. 

195 

Access to manufacturers * reports is important, e.g. 
Westinghouse RCA etc., have reports and are valuable as 
they show in what direction different manufacturers are 
proceeding with unclassified research. Also those 
published through the Office of Technical Services, US 
Department of Commerce, Washington DC, US Govemn^ent 
Research Reports, atuJ Technical Reports Newsletters etc. 

Writing on the "reluvance return" form about changes in 
requirements seems a very reasonable way of doing this. 
However, when T notified you about information on electro- 
magnetic screening only a few articles were listed subseq- 
uently. Of course, this may well be due to the fact that 
only a few articles occur but no notifications seem to 
have been made of articles in the IEEE journal on 
"Electromagnetic Compatibility" • 

Also I seemed to got an increasing number of notifications 
on integrated circuits, both usage and fabrication, most 
of which I marked "X". I suppose it was due to my lazi- 
ness that I didn't notify you about their non-relevance 
to my personal work (probably a paying customer would) 
but I think it would be to your advantage if your system 
could have detected an excess of "X"s on a particular 
topic and quizzed tlio customer if he wanted this topic 
deleted from his profile. 

197 

I have concluded wilh regret that, for myself, the service 
provided by INSPEC Is superfluous. 

This is possible due to the fact that my main present 
interest is in medical electronics. In tliis field the 
majority of information is presented in a relatively 
small group of jouriials and periodicals and it is 
practicable to maintain awareness by scanning the original 
source material - wlilch had to be done anyway in order to 
provide your relevance returns . 

In this somewhat compact field of interest , moreover, 
highly relevant arti.cles are frequently written by workers 
who are known to oneself or associates In the field of 
interest. It is moro satisfactory to scan the specialist 
periodicals and journals (including editorial material) 
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rather than a list ol tlfclos whicli Trequontly convey little. 



I have approci.aLod your eTfortrs and re^^rot the. need for 
the above; cri t ic i.sms . However, they concern the Tunda- 
-mentals rather than i he iinplementatlon of your service* 
i have personally Tound more useful those systems which 
provide information ctrieval with retrospective references 
on a specific topic. 

199 

I can suggest only one improvement - the modification 
of the cards so that a code is available for the user^s 
retrieval of particxilar subjects from his own accuramulated 
stack of cards 

201 

-In fairness to this investigation, my proiile is hard to 
satisfy using published material in periodicals • 

Research work like mi.ne is pocketed and information 
scarce, but in the other subjects like lasers the information 
could be overwhelming and the service could tailor and 
select the relevant information* 

202 

In agreeing to participate in the investigation I recog- 
nised that my interests were somewhat marginal to the 
field covered by the experiment and did not expect much 
return. My expectation was in fact proved so I believe 
that my comments deserve rather low weighting in any- 
overall assessment* 

206 

I have been very happy to co-operate in this investigation 
and 8im glad to have been of service* My own particular 
requirements have been well served except for the number 
of reports not notified because they were not covered by 
the service* Even i.P reports were included there would 
still be a gap because many reports of potential interest 
to me are classified under security regulations. I see 
no easy solution to this* 

The service has resulted in my be i,itg able to make more 
effective use of the time 1 can spend scanning and reading, 
i.e* less scanning, more reading. 

One aspect of changes in profile. It would be useful to 
be able to backtrack through the system to bring out 
previous articles which woul have been rejected under an 
earlier profile. 
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1. shall not be ssibscr ibirif? because my own "library service 
is sotti.ru; up its own system but Lf I were back in an 
industrial or pri.vai.f capacity I would certainly favour 
the use of the service. 

If 1 can be of further service at any t Lme please feel 
free to ask« 

208 

Many of the references wore very wide of the Information 
Requirements submLttod in November I967. Most were highly 
theoretical whilst I tried to Indicate a more practical 
bi.as. Perhaps 1 should have asked for my profile to have 
been modified, 

209 ^ 

Our su/:/;estion (for your obvLously "scientific" clientele) 
is that you mit-ht LiidLcate how profile can be altered - 
i.e. I feel that T want the search to be very wide but can 
probably suggest how to restrict It better than you can 
(With respecti). For instance, if I know something about 
the logic I could specify "Spectrometers, + Mass but not 
Spectrometers Radio Frequency" or similar combination~ ■ 
It may be that one could use the selective system more 
cffectLvely with nioi(! knowledge of its method. 



212 



If notifications can be kept to within 2 - k weeks of 
publication the service would be Invaluable. 

221 



The results of the investigation 
when allowance has br-en made for 
my profile. 



have been useful particularly 
the specialist nature of 



219 

A lower cost would fx- essential if our own information 
services are not to try and provide a service of their 
own for wider use. 



220 



The service was treated as a separate exercise, ignoring 
our own Culham li.braj y service which is good but not so 
good In presenting information on cards and relevance. 
I am not justified oconomi cally in continuing the service 
on just this basis. C feel, although I knov that my 
reference system will now be much Inferior to my present 
card index obtained over the past two years, greater 
coverage would probably have weighed the balance in your 
favour. 
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The Culhani labora tory ( my employer ) provides a s imilar 
service but I found the SDI Service better and more 
convenient. In general it required less time to sort 
through and was slightly quicker. However, these 
advantages do not justify continuing with the service 
on a cost-recovery basis,, because I can get the same 
information (with slightly more effort and a little 
more delay) from the Culham Laboratory Service. Also 
the' Culham Service Includes major foreign-language 
periodicals . 

242 

The major snag i.s thai; it is "often impossible for some- 
one else to say whether a paper is of interest. As I 
mentioned previously words 1 Lke "Breakdown" "Discharge" 
"Separator" "Spectrometer" are used in such a wide range 
of senses that Irrelevant material is inevitably turned 
up. 1 would have thought that a computer service of this 
knd, once set up would cost less than £35 per head. 
Though I suppose this depends upon the number of par- 
ticipants. 

j; am glad to have been able to help you. 

I think the SDI Service would be useful to those who 
do not have their own information service but I am very 
well provided for in this respect. 

It would be very good If, for very relevant articles, 
you could supply a copy very quickly. 

I hastei.i to add that this does not affect me too much 

as we have a good li.braary Information service plus access 

to AERE Harwell Library. 

The accent Is on relevant, the person requesting would 
not have to request willy-nilly. 

In my case notiX*ica t i on of relevant conference proceed- 
ings and reports would be very useful. However, here 
again the library do a good job on this. 

The main problem as I see it Is as follows: in my office 
is a stack of periodicals, trade journals, proceedings, 
reports etc. Two major problems stare you in the face: 
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1 To keep reasonably abreast of current developments 

2 To obtain copies of very relevant articles pertain- 
ing to the project in hand and to pin-point them in 
the mass of material. 

to keep up-to-date you have to struggle with an expanding 
fxeld, so the information service must be more selective 
and precise. If the participant can feedback guidance on 
thxs then very relevant articles need not be missed. 

One main advantage ol' the service for me - it keeps me up- 
to-date reasonably wnll without having to plough through 
several periodicals, per week. 

My local library arc considering my request to stay with 
the service . 

246 

Many thanks for the Information supplied which I have 
found applicable to my requirements. 

As my organisation Ls Government controlled and has very 
good library faci.lities, i.e. Rutherford and AERE, it would 
be, L consider, an expensive information service consider- 
xngthe number of peo])le who could use it. This is the 
main reason for my not wanting to continue with it. I 
have discussed this with our librarian and suggested the 
service could be centralized through them as most pub- 
lications can. Generally my profile is electronics which 
offers a wide range of systems therefore could be cir- 
culated to a number of people within the establishment 
for economic considea^ations . 

248 

J think the accuracy of the profile will be considerably 
improved when authors are encouraged to compose titles 
from words taken from a standardised thesaurus. 

In my own circumstances, supported by excellent library 
and information services, no real advantage can be claimed 
for SDI. Over the yt^ars an "entente" is formed which 
is invaluable. 



Again, from my own p6int of view, I regard foreign material 
as essential. .1 can road fluently in French or German 
and wish I could handle Russian. In Fact I have just 
started to study Russian and hope to reach a level just 
high enough to translate titles or abstracts. 
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I carmot answer Question 5 without making an extensive 
search but, Trom memory, the majority, if not all, ar- 
ticles or special Interest were notified by SBl. 

As an experiment you provided a page printout in addition 
to the cards for a l^yw weeks. I think I replied that 
1 preferred the cards for my person use. However, I have 
had second thoughts about this and while I still prefer 
the cards for my own use I now think it would be useful 
to have a page printout In addition. This woul have 
the advantage that I would still have the cards for my 
own file but that I could circulate the page to members 
of my group to draw their attention to articles which 1 
think they should soo. 

Be.cause my field of work is on the edge of electronics 
much of my reading Is^how In journals which are not with- 
in the normal scope of Electronic3 (or/even Physics). 
Also, quite a few oP the relevant papers are in Foreign 
publications. Personally, I will revert, to (or continue) 
scanning the current papers list, plus other more specialized 
abs tract services • 

However, a joint profile has been prepared, in associa- 
tion with F F Roberts and R F3 Wyott and in this way, I 
hope any Items I miss, will be detected in this way. 

262 

I find scanning the relevant sections in CPP and CPE 
more satisfactory* 1 have found many articles of in- 
terest (many of them foreign) that have not appeared 
on my SDI list. 

264 

I feel that the service has met my requirements very 
veil. I particularly like the cards as I can use these 
for requesting loan of items from the library and for 
recording conmients on after having seen the article. 

267 

I don't yet feel 1 can rely on the service to the 
exclusion of oLlior mc^aris of inTormatlon retrieval. 
Consequently most oV the articles I have already spotted 
before notification. 

The cards are not smTlcently uniform in .-3ize to form 
the basis of a card index. Tills Is the most serious 
drawback of the scheuie. The sJze should be standard 
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(do riilernational sl.cmdards exist, I wonder), and users 
sliouid bo provided with a supply of blank (or lined) 
cards, so that a cai*d index can be formed. 

For the same reason, the layout of information on the 
card is seriously at fault. I have previously given 
proposals on iinpr.pvoment , a:ad so will not repeat them 
here. 

A broad distinction in electronic en^ineerin^.^ is bet- 
ween linear and non-linear circuits. I have no interests 
(at present) in non-linear circuits, and this was given 
on my profile. Yet I' frequently received non-linear 
references . 

Broadly, I feel that SDI is a brave attempt at an almost 
hopeless task. 

262 

I prefer the monthly CPE. 
221 

Subject and author searches in retrospect would be a 
useful facility. 

222 

In a three month sample period, I find I filed ei^ht 
references per montli of which three were SDI cards and 
five manuscript cards. The reasons why there were no 
SDI cards in place of' the .manuscript ones were:- 

1 Foreign-language journal 

2 Not electrical engineering 

3 Internal report 

4 Probably outside period of SDI search 

Thus there seems to bo no serious failure of the SDT. 
system as at present organized. However, the advantage 
offered is marginal at present because :- 

1 Useful SDI referonces tend to occur In only a few 
journals which can and usually are, quickly scanned 
anyway . 

2 The need to obtain references outside the SDI field 
remains, and involves fairly considerable' effort. 
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On^Jancin(j throii^jh a periodical which contains an article 
which has been selected by my profile, I occasionally see 
other articles which are equally relevant and have not 
been selected* An example that has first occurred is in 
IEEE Journal of Solid State Circuits for Juno 1969. I 
was notified of articles on pp 166, 122, 110, and I3I, but 
do not seem to have been notified of the articles on 
pp107, and 1^5 which are also relevant. 

(N.B. On looking through the cards I see that I do not 
have any in the range - maybe a batch got 

lost )• 

Arising out of the above, it would be easier to check 
whether a given article has been selected if the card 
numbering have some relation to the periodical or the 
article, i.e. if all articles in a given periodical had 
consecutive numbers. 



In the case above the article 



221 



on pllO is numbered ( 052)079 
pi 22 " 078 

PI3I " 122 

pi 66 « 072 



I have spoken to other users, some of whom have had trouble 
getting their profile to work. Tt vould seem desirable 
that (a) keywords continue to be listed on notifications 
(b) information of a general nature be given to new 
users as it becomes obvious (after a while) that certain 
keywords are always present in irrelevant notif icc\tions . 
(e.g. in my case I need synchronisation and not microwaves 
to reduce jurik content )r Such ideas occur to users who 
obtain information on (or guess) how the SDI system works. 
My profile is now hightly satisfactory. 

The subject of Telecomras attracts authors writing 'in 
Swedish and Japanese-, as well as English. Many of the 
Swedish papers appear as Manufacturers Journals (e.g. 
C H Ericcson) but the Japanese are more general. These 
later have often got English abstracts, which with the 
diagrams and equations of the original are highly in- 
stinctive. These ought really to be included in you 
cover. Papers in German, though less frequent, are 
equally important. So, please, lets have foreign cover. 



Service is too quick (or our library is too slow) - often 
several weeks delay in obtaining U«S. published articles. 
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( ^lestion 1 

About 25 per cent or the references received during 
1969 were considered to be not relevant. This was 
mainly due to difficulty in distinguishing between 
the purely scientific and practical applications of 
various papers on the ionosphere. The percentagv5> of 
references which did not match my profile in any way 
was probably about 5 por cent# 

Question 7 

I think that my needs concerning foreign language papers 
are not at present by the available English translations, 

Question 4 

In future the SDI service will enable me to 
reduce the need to scan publications to help myself up- 
to-date, 

284 

I tend to want information on a specific topic on odd 
occasions. It would be usefule on these occasions to 
request a 'search' for papers on this subject, a fee 
being payable on a 'per search' basis. 

286 

MOst articles which were of particular interest were 
seen without use of SDI service. On few occasions when 
references were sou/;ht little new was found and in one 
case the reference i;o the information required, in an 
article quoted was very indirect, 

I feel very stronly that to be of real use the notifi- 
cations must include a synopsis. This was done one week 
in November I969, card reference No, 962493 and was a 
great help. The very general title of articles makes 
it impossible to know what the article is about. 
An example is card 9^701 9 '»^is txribution of the number 
of crossings". Prom the key words given it would seem 
to have something to do with frequency modulation detec- 
tors but from the title it might equally well apply to 
railways! There is no indication whether the article 
is an exhaustive study or simply general comments. With- 
out this information much time could be spent obtaining 
article which subsequently proved of little interest 
because of the treatment given to the subject by the 
author, 

m 

A ereat deal of partially relevant information is coming 
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through at the moment. To some extent this depends on 
how strictly the ol\ferings are judged. Most references 
have some relevance, and nearly all add to backgrbund 
interest, and some that are partially relevant now could c 
be highly relevant Ln two years time. 

I imagine this describes one of your basic problems 
and in the past I remember opting to risk receiving an 
excess of partially Irrelevant references to be more 
certain not to miss the occasional highly relevant one 
which the computer may be in two minds about J 

Despite a surfeit of 2»s the service has been highly 
appreciated. 

221 

What about considering printing the Authors Abstract 
either instead of or as well as the keywords. This would 
help one sort out the fringe interest papers or the ones 
with titles not correctly advertising the contents. 

In general some unwanted articles in the notifications 
are a small price to pay for complete certainty in finding 
things . 

I am sure the success of the service is due to the con- 
tinuous feedback of 1, 2, X, assessments and this should 
be kept up if possible. 

Although I spent mox^e time on reading I gained much 

more useful information and the time was more efficiently 

spent . 

In question 7 I am not sure whether "learned journals" 
come under the heading of periodical, if "periodical" 
means less learned publications, such as trade journals, 
or popular review journals, such as "Scientific American", 
my answer, would be 



297 

The service would bo par ticulai^ly valuable to people 
having little or no access to library facilities. Here 
at RAE, with an extcjxslve library service, the main 
benefit of the SDI service is that one is saved the 
trouble of looking tlxrough indices for relevant ar- 
ticles of interest. 



I think that where time is at a premium and where people 
are working at a distance from a library reading room the 
service would be very worthwhile. 



Thank you for the information sent to me over the past 
two years. 

202 

I think you have been very successful, there is nothing I 
can think of adversely. The information I received showed 
early the nature of the service, I would have been com- 
pletely unaware of most of the matter your notifications 
brought to my attention. 

It should be stressed that a few seconds scanning your 
notifications brought to my attention matter that if I 
had obtained this from perusal of publications would 
have kept me so busy reading through the number of pub- 
lications I would have had to see to find out the same 
number of references, I should have had little time 
left for work. Tn addition I have a card reference of 
tremendous value that I would not have compiled myself 
aid this will continue to mean a vast saving in time in 
future when I want to see if something has been written 
or a matter I want information upon. 

Widely used this service is of inestimable value nation- 
ally. Availability of information so easily has a second 
benefit that newer ideas and techniques are encouraged 
and much more likely to come about. Congratulations. 

303 

In my answer to 1 1 I. should like to include that my local 
library/information service provides what I consider to 
be an adequate source of information. In consequence 
it would be very difficult to justify the cost of the 
service. 

On the whole I have found the service satisfactory. I 
feel that my cormnents on too many irrelevant articles 
and profile control are as much my fault as the system, 
since I could have asked for my profile to be modified 
but did not find time to write to you about this. 

m 

The title of most articles does not convoy enough 
Information. An abstract is needed othei^wise one still 
has to visit the library to obtain and scan the articles. 

We at RAE are supplied with at least 2 selective info, 
services, one at Establishment the other at Departmental 
level including reports etc. not normally available to 
the public. These services are extremently efficient 
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(alfchou/vh probably vory expensive to run). Any individual 
or orfranLsafcion however, without this payt of service 
would benefit from .SOi: provided the pei^ceiitage of relevant 
to i-rrolevant references increased. 

222 

1. foel that the service was very good in fjeneral, the 
perturbations that occured when the profile was chan/jed 
tended to frustrate, but when stable, tlie return was 
Cood. With a larfre LLbrary and excellent facilities withLn 
the ostaijlishment, if, was felt that wo could not continue 
the servLco. 

211 ■ 

The cost oC the service i.s too h i ;vh to convence manaf-einent 
ol tlio savLn^s i.n time which iiiL^iht happen. Money and time 
are not alway obviously interclianf^eable. 

F cannot surest what: pr Ice-f i,-xire would {-.aLn maximum 
responso. 



As 1 liave stated previously that some titles tend to be 
nusleadin^r as to the subject matter of the article. Some 
time IS spent look in/-- up the article before one is able 
to assess its relevance. This is not your fault and i 
don't see how this problem can be overcome. 

7»lthou/ih i: prefer cards to computer printouts from the 
fili.n/.r point of view, tliere is a personal problem of find- 
i.n,r Viiinfr space. Since the service }ias been in operation, 
the cards collected represent a considerable volume and if 
one multiplies this by a yearly factor and bearing in 
iiund tJie expansion of tlie service then the problem from 
my personal point of view would yet worse. For this 
Kason ]• would consider a f or tni/yhtly or ever a monthly 
service mi/^ht help to ease the problem, or better still 
if my central library lield the set of references only to - 
wliich we all liad access. 

1 think that the way to /:et most out of the KDI service 
is to adjust the scope of one 's prof ile so as to cover 
say 90-95^ of very relevant articles. If the profile 
is too narrow, clearly a l&rcc number of articles would 
be missed, and if too wide tlie selective part of the 
service is lost. Therefore, the use of the service 
depends on how careful you are in specifying; your pro- 
file. In all information searches there are always 
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articles on Lho fViru:o on onc»s prori.lc whLch could be 
Vf!ry rxvlovant. I do uoi con.sifJop It tiic Job oP an SDI 
«ervico (,o cover Lho«o. 

As a means of obtain Ln(j a hardy XLst of references the 
service works admirably. 

One /^reat virtue L have found Is that receiving: those 
cards early enables one to «et to tlie head of the queue 
of people waLtin^j to obtain the article from our estab- 
lishment library • 

X feel the inves t Lfja t ion has been carried out Ln a very 
well or{;anized and useful manner. 

My (general conmionts are necessarily influenced by tlie 
{jood service provided by the Ru'lD library. SDl noti- 
fications has sometimes been quick, thou^vh this is not 
usually of sicni f i cance . 

I have picked up a few articles which otherwise would 
have been missed - say 5^ at most of known relevant 
articles. The time involved in taking part In the In- 
vest l/,-ation has been or ne(;li^rLble significance. 

-m 

As a designer of electronics cii^cuits it was difficult 
to write a suitable profile, and T find there is no sub- 
stitute for scanninr; Hritish and American Magazines and 
manufacturers' literature. Often, items of interest are 
found in the most unlikely articles. 

The profile was deliberately made broad, but this meant 
that a lot of irrelevant information was received. What 
mUsht have been more useful would to have been able to 
hr.ve asked, once only Tor all information in the past 
(say) six months on one particular subject (e.g. appli- 
cations of MOS digital Integrated circuits, or 'the use 
of PETs in radio frequency amplifier circuits.) 

Thank you for tJiis service. 1 understand from our ii binary 
that a group subscription may be organised for the division 
at RRE (Division L1 ) in which I work. I shall be leaving; 
RRE in August 1071 to set up as a private Consultant, and 
I shall probabily then wish to Join your SDl scheme on 
an individual basis. 
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I i.hifik i I, woulii bo hf-tioncial Lo tiavo a botLor system of 
protlur.inr. a prorilc. Ono of th<: koys Co the relevance of 

inroniial.iun rucoivod is tho profile. If -this is too vide 
and not spocLfLc one receives a lot of Irrelevant infor- 
iiiat Lon. 



I would su/i/rest that a prof i.lc- form be produced in question 
format. At least i.t should be pointed out tliac a prof Lie 
should be specific. i 



ate 



I 



in my particular case?, althouj;h lists are /jonoraiiy relevant 
to my requests, my true requirements covered a wi.de ranf^e 
of subjects whlcli do not usually appear in periodicals," 
furthermore, it would not have been possible to specify 
at the be^rinnin/r of, say, a year what subject i: was read- 
.ln{; to cover. 

Questions 

Initially i: found my profile was biased too much towards 
physics of thin films. Y mentioned this as a criticism 
at avery early staj-e, but it was only after many months of 
mxsses and near misses that the profile appeared to be 
reject injv this type of article. 

Question 10 

Value of service to mo has particularly been In speedy 

notifications of rerovant articles. This has meant L 

have been near the top of any waiting; list to view articles. 

The service has been a useful "back-up" to the local 
library lists and abstracts. The SDX service appeared 
to scan a wider ran^o of periodicals than do the local 
library lists. 

Most of tho ar^tlcles which were missed seem to have come 
from specialist computer magazines in particular the BCS 
publica t i ons • 

1 do not know whotlier we are unusual In liavinr; (a) a /^:ood 
library, wi Ifi easy access to other libraries, (b) a /^roup 
wlio manaL^e to keep an eye on the literature, but for my 
part the SDL service, althou(5h excellent on the whole, 
was not particularly useful, in that L had almost invariably 
seen the relevant papers, by the time SDX notifications 
arrived, or noted them, for future reference. 
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T 



I think there wore about three Lnterestin^j looking- papers 
which r had not: scon and which were readily obtained by 
the library; onJy l\^o of these were in fact relevant* 
Two were In journals we do not receive • 

I did mention several times that 1 was not interested 
in plasma physics, etc., at the be^^lmiin^j; my interests 
are a) R.F (in the field) measurements 

b) Instrumentations of lar^je anteimas* 
Nevertheless E received a lar/^e number of papers which 
fell Into neither cato^vory, no doubt throu/^h an accident 
of codUwr, {por example althou/::h I am interested In 

l^C' semiconductor devices, once they are a reliable 
C'oi.nr; concern, I am not Interested In papers on obscure 
effects, or how they depart from expected physical mech- 
anisms, and, {since we do no development work) in knowing'- 
how to measure tiieso peculiarities, T am not i.nterestcd 
in plasmas, propa/^atlon In them ar aerxal impedances in 
them) however, this may bu unavoidable and is not a great 
inconvenience, merely a waste of paper etc. i.'i (They 
just went strai^;ht into the W.P.B.) 

2^ 

It seems to me that In my particular Tield the headin^js 
under which the articles are assessed for relevance are 
rather too broad to ^-^ive a \'ery fjood service, (Obviously 
the number of fields Into which articles can be sub- 
divided is limited, and this I Ima^-ine is one or your 
problems). On the assessment of prof lies generally, I 
wonder If it would bo worth allowing users to see a list 
of headUw-s (the ones listed on the cards, ew:* Ionosphere, 
Radio waves. Frequency LF, etc.) and help to form their 
own profile from this sort of Infoini ation. Ft would in- 
volve more work on the user's part, of course, but cuts 
out one sta/:e In profile assessment. 

I'm sorry not to be continuing with SDI Service, as I 
think it is a very worthwhile exercise, but unfortunately 
my employer will not support it financially and I am 
not sufficently affluent to do so personally. 

Thank you for your servi.ce over the last two years. 

2^ 

My comments have been Influenced by the fact fJiat T am 
worklnfj in a field where researcli is lon/^ term and where 
rapid chan/^es i.n outlook, theory or experiment do not 
usually occur. So a vei^y rapid service- is not essential, 
r also }iave easy access to a /vood library and information 
service and mucli Information is obtained by contact wi tli 
other workers. 
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ERiC 



It I.S preferable to mo to have too many items, including 
irrelevant ones, than to miss potentially relevant items. 
Many items can be relevant too, as background information 
rather than being directly of great value. 

The main problem in using the service is to know whether 
to rely entirely on it and really reduce scanning time or 
to mistrust it and scan anyway. Howedver, I have found the 
service aseful as yet another method of coping with 
publica^-lons and the time spent scanning the cards them- 
selves is very small. 

I shall probably be Joining a grcup subscribing to the 
service (when we can agree on a suitable profile). 



Over tho period of the survey I find that m>' profile has 
gradually changed and that new aspects of my work need 
searching back over a number of years to find the relevant 
information, thus SDl cannot cope with this aspect. 

I find in general that since reports were not covered, I 
have seen much of the important information long before 
It IS published in journals. 

Much of the more important new papers such as in Applied 
Optics, Rev Sci Instrum etc. I see at about the same time 
as the cards arrive. 

2§o 

The main limitation to the value of the service is the -Pear 
that some relevant articles will be missed, even if this 
few can be shown to be unjustified. If no articles are 
missed there is no need to scan the literature; otherwise 
the whole scanning exercise must be carried out and the 
service has achieved little. 

On the other hand the service is very useful when one just 
doesn't have the time to go" to the library for a while 
because 

a) it is highly likely to give reasonable 
early warning of the most relevant 
articles and 

t>) it jogs one's memory that the time for 
a library visit is nigh 

Although a limitation to the present scheme and not so 
convenient for card indexing systems, the cost of the 
scheme might reasonably be reduced by asking each user 
to name say six journals that he would regularly scan and 
receive no notifications thereto. (90 percent of my 
notifications have'ibeen from six journals.) 
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Many tlianks for the sorvLce you have provided, I have 
found it most usoTul. 



Two main limitations were:- 

a) no foreign lan^^ruage journals 

b) coverage did not extend to journals (such as J applied 
Meteorology) In which papers relevant to radio pro- 
blems now appear. 

262 



The main problem is Lhat no computer can pick out papers 
which appear to be "way out" but which may trigger a 
train of useful thought. Thus C feel I will often need 
to scan the main journals. 

One aspect whLch i: have found useful is the availability of 
filing cards. Even If I did my own scanning entirely these 
cards would save a lot of time as I previuously produced 
cards myself for a filing system. 

The service has saved me little time because, in its 
exper Linen tal stage I was .not prepared to trust it not 
to miss any article of importance. It has proved to be 
very reliable and I would now be prepared to rely on 
it with some confidence. 

I have found your staff most helpful and efficient as 
individuals, but somewhat impersonal as an organisation. 
This may be unavoidable. 

368 



The service increased the flow of information to myself 
and my staff. It also increases our work load to the 
extent that I have almost cancelled it, however, it is 
valuable. 



Ji§2 

Towards the end of tJie investigation L changed my employ- 
ment from a inLstry of Technology research establishment 
to a university. However, In both cases I was responsible 
for the management of a computer service. Thus L had a 
general interest in all articles on computer techniques 
both hardware and software: when such articles were not- 
ified to me I graded them as 2. At different periods ia 
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<.l"- .ny..-.sl..,:aLion I spoci.ri.c major Lntcrcsfs depending 

/rtlcTc '"■^••'^^^^''f '•■-'-'^•^^^ on the compute? 

A Lxcios on such subjects I graded at 1. Unfortunately 
a i received further notifications on such subjects tL 
project concerned mi^vbC be underway (i.e. the programs 
wrxtten) thus the grading might reveit to 2. J 
would presume led a rather unstable profile in my case. 

Lrrtn ^t^"" *° -y°" Tor. the opportunity to take 

pare J.I) th Ls mvos 1. 1 <va(, i on. 



J77 

nospito (.ho fact iUnl my replies coi.taLn some cri t Lc i s Lmc= 
roco,.TMso that my profile wou J d bo very di.fricult io 

tn e?o '° ^^^•>' -^^^ spread of 

••ntoiosts covered, and the fact that i; am not emra^red In 
purely 'research' worlc. en;, aged xn 

379 



service which 



On the whole I am ver-y glad to have this _.v... w„xc 
imposes very li.ttle ei'fort in exchange for a regulir xn- 
put of ..nformat.on. 1 do not expect Lt entirelj^ to 
replace a re^vular browse through the library but it re- 
duces any anxtety of missing important articles if r do 
not fuid txme for this. 

380 

My requirements for the coming year will be covered by 
the subscript i.on of another person within the same group 
wxth very sxmilar interests. t>roup 

381 

I can only say it has been of great use and J am con- 
ti.nuLng wi.th the servi.ee, many thanks. 

382 

From the type of article notified, I wonder If the pro- 

Jai: f;rr;iirSec\°?-^°"'"°"^"" ^^^^^^^ --^^ °- 

ArLLclo conLciiit: RoquLrQinonL 
Osc ilia tors 
M i crowavos 

Semiconductor DovLcos 
Thyris tors 

This article would not be wanted because it treated the 



riiyrisLor control of iiiicrowavo ci.rcuiLs and was a low 
I'n-nui'.ur.y sul>JocL, an.i so nol wantod. This may not be 
an idoal example but on a number oV occasions some 
notation would dan/^er of missin/j something useful by 
this technique, so it may be used only continuously 
perhaps. 

m. 

I understand ti.at soloctlon was based on a match between 
profile and a sol of keywords abstracted from the article. 
Some of th.ose which turn up frequently arc ambi,.;uous (o.r. 

correlation") and otliors convey zero information (e.r. ' 

noise") since they will appear In virtually all articles 
in say tho coimnunica t Ions field. 1 feel that titles form 
a bettor basis, and my jud^'ement of relevance was, perforce, 
mai.nly based on (Itlos rather than readin/- the articles. 

1. prefer our local system of circulatin/j copies of contents 
lists or periodicals, si.nce 1 know which journals to {^0 
for and can scan a pa;';e much quicker than umpteen cards. 
Any article missed will if it is a cood one, be cited some- 
where else soon onou/vli for my purposes. 

Quite honestly, i do not think it will ever be possible to 
automate human communications successfully. 

The service has been very useful, and has brouRht articles ' 
to light which might otherwise have been missed. 

1 think it would be useful if the profile could be mod- 
ifxed frequently (i.e. every month or two perhaps) to 
look for specific articles besides those on the main 
profile. This is envisaged as a sub profile which could be 
tacked on -no riio Kci-lsv i.-rofilc. 

This might help tliose who are engaged in a wide field of 
research but require specific information at a given time. 
Have other users similar views? 

m. 

The only serious omission in the classification seems to 
be uhe division of networks and circuits onto classes, in 
sterms of the frequency range in which they are Intended to 
operate. 3- have regularly received a batch of references 
Lti which 30^ refer to work In the microwave field, which 
is of no Interest currently. Possible this coulc^. be over- 
come by modifying my profile, but J. deduce from the "key 
words" on the bottoms of the abstract cards that- it may 
not oe easy to subdivi.de by frequency of operation. 
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ERIC 



On (.IK! wliolo Lho rraiii«;work iiius L bo Loo rif/ui for tlio 
development areas in wlii.cli ! work. I.uLerosL areas often 
chan/je rapidly and on Llio e<Uve of /..overnjiient work are 
rarely if ever reported in coimnorcial journals. 1 seemed 
to /'ain more from our library information service than 
from SDI even in areas J would have expected my profile 
to cover. 



2§1 

Many firms, universities, etc. have card sortlnf.? machines 
these days. At a small additional char^je, 1 am sure that 
tlie cards could he usod for Information retrieval as well 
as dissemination, and the usefulness of the service would 
aj /'.Xf-atly Incroaso 

b) be hotter than (.hat of rival or/van Isa t Ions e./j. 
Lowry-Cockrof t Cards 



J27 

I have found tlie SDI service nonerally satisfactory and 
the main complaint be in/; the numbor of Irrelevant ref- 
erences - especially in the early days - this considerably 
amproved by tlie end of the operation. 

The usefulness of the service to me deteriorated due to 
a chanft-e in circumstances which had nothin?; to do with Uie 
SDI scheme, but as occasioned by the well stocked library. 
Thank you for let tin,; me enjoy the SDI service for such 
a lon,j period and 1 hope I have been as much lielp to you 
as the service has been to me. 

22^ 



Tho profiles would be better in my view If they were 
more selective. Of course, initially one did not know 
how tlio scheme would work out but the user could, if 
startln/; a,;aln, make Ills profile more selective by reduci 
the field of interest somewhat to his speciality. lie 
would then rely on the occasional assessments of articles 
more heavily to catch the ( inevitably greater ) number of 
mined articles. 



221 

I have found the service exceptionally useful and It has 
certainly solved the problem- of current awareness. There 
remain two problem areas which mif;ht be incorporated Into 
the service eventually. 

1 Searches, of published literature. My field cuts across 
many established disciplines, so I often have to look 
Into papers on some we J 1 -es tal)l i shod sul)j<-cts. This 
would also help people starti.n/y work In any field. 

2 Papers can not be read thorour;ljly quickly so unless 
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Uio Ml, Parian is very f.oloranl, oiuj must acquire a Xerox 
copy or- a roprini. . 

Til is, of courso, i;.;noro« Llio hir.fi'^-sl probioiii of." all - 

f indLn;r L i.ino to read all Lho papers of intoi-osL. Thank vou 

tor maki.n;j: it a littlo oasi.or. 

hn 

I. {jet the foelLn;.-; tliat, had my Job interests remained largely 
unchan,rod dur i.n,.; the i.nvest L,rat i.ons , the items would have 
been on avera;.vo more relevant sti.ll. t think that my rel- 
evance indications arc partly coloured bv my personal in- 
terests, as well as tJio interests demanded by my professional 
interests, which in fact chan^jed .in the two years or SDI 
quite sMvnif icantly and mom than I would have suspected 
two or threo yenrs a^^o. L thinJc this chan«intr pattern of 
interest ]ias probably confused the determination of my 
particular profile. 1 suppose this type of chan^^e is 
.LnevLtablo and quite common, so presumably the only remedy 
.IS supor-rapid response of profile shaping to chancinc" 
relevance indications, i.e. week rather than half a year. 
This is not to say that X think it takes half a year Tor 
a profile to respond, X. do not really know how lonf it 
takes. 

hQk 

This would be a useful service Tor me to have as a supp- 
lement to existin^j library services, especially as our 
firm provides no abstracting^ service such as was provided 
under former mana^jenent (no doubt on cost saving ,^rounds). 

Over tJie past year 1 liave done no work in accordance with 
my profile, having beon transferred to a team re-equippinr 
Goonhilly f Aerial, and it is only in the last couple of 
weeks that 1 have got back to pulse work for which my pro- 
file was formed. 

I cannot imagine any design engineer in my company spending 
even a penny of his qwn money on your service as salaries 
in real terms are going down, not up. 

i: think your service would be extremely valuable to res- 
earch people witli a narrow profile and wish you luck in 
convincing those who hold the purse strings that this is 
so. 



r. find it not possible to answer Section 11 as set oui 
My reasons for not supporting the service now are two 
fold : 
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Ihii vory lii fjJiLy .spec i.al i.sod field Z am working in is 
noL yet covered by many publications and calls on a 
vory 1 united ar><-a of the microwave measurement field. 

I. consider that the onus of paying must re«5fc with the 
employer in general and he is unlikely to support an 
individual unless liis particular field is widely pub- 
licised and SDI could therefore, save much search time 
or load to rapid updating in a fast moving scene. 



'no 



I. am slightly miJiappy about tlio way In which the original 
profile was formed. Since the efficiency of the whole 
system seems to depend critically on getting the profile 
right, perhaps wo should have boon more careful about its 
Initial compilation. 

it Is rrot always clear to the user that the profile gen- 
erates the "net" In wii Ich relevant articles are to be 
caught, and that failure to be sufficiently specific in 
Its generation can cause a rather side mesh to result, 
producing lost articles. For example, my profile did not 
specifically mention "radio wave propagation" L had 
though tills wonld have been covered by other more gen- 
eral key words - and as a result 1! have missed a number 
of relevant articles. 

Are you sure that the instructions sent with the blank 

profile are sufficiently clear on this point? Surely 

the moro detailed the profile, the better for you and us?. 

As far as i can recollect I received no Information on 
Conferences, Books, Patents, Manufacture's lists, 
standards and specifications. Theses and dissertations, 
Keports, In English, let alone foreign languages. T 
have marked these items as in question 7. A lot of very 
useful information comes out of conferences. It would 
also be vory useful if it were possible to carry your 
literature search to earlier years. Particularly in 
patents which frequently contai.n information not available 
in pori.odLcals from conferences. 

inO 

Apart from a subscrlljors normal service in the subject of 
his profile, it sometimes becomes necessary to investigate 
a problem quite outside his normal line of research. Are 
facilities available, or could they be made available, for 
an individual extraction of articles on any particular 
subject? 
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Tl»o s(!rvi.co has onablMcl mo Lo make Tar more elTiclenl; use 
or Uie l.lmit.od time I have available for Information 
gatherin/^ and as such it has been extremely valuable. 

h2k 

On the whole a satisfactory service was provided, and 1' 
should like to tJiank you for this. 

Now that the SI) 1 Service Is on a cost-recovery basis thore 
Is a need to make a more bus lnoss-1 Ike approach In de- 
V\i\in(r what the service can do. \ should welcome a report 
on the Inner workings of the system, written for the bene- 
fit of the user: crj it would bo helpful to know if one ca 
interact effectively by usin{^ key words to encoura^;e or 
discoura/je certain types of reference. 

^26 



The addition of some ''hi/iher-level" Key-words (such as 
"broad relevant" V^eneral principle" (i.e. application 
to other fields), "authority" etc.) thou(^h admittedly 
(ilfficult to us In indexing might pay higli dividends by 
filling in gaps which a more "mechanical" program might 
leave. This would tend to minimise the difference between 
a computerised service and a direct person to person one. 



I would like to thanic your for the service px^ovlded during 
the last two years and hope for your success In the future. 



J tliink that a worthwhile service has been provided, es- 
pecially in view of the ever accelerating volume of pub- 
licised articles, and I hope to benefit from It in the 
future. 



I am a little worried about the apparently lii/vh cost (£^5) 
for the yearly service, although f can undcjt^stand that th<j 
cost of the computer services must be high. 



During the pas t year 5O or so periodical articles were of 
value, but at tho very best these can only point one in 
the right direction. Most 'of my problems liave been of a 
development natui-e and therefore the only real solution I 
to "suck it and see". 
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I have a foolinfr that this type of excellent services 
WLll bo of <^reat use to those in pure research, but 
Limited to those lessor mortals, like myself in industrial 
research establisluiietits. 



I think I. should have put more effort into making my 
profxle exclude circuitry articles, but 1 didn«t"'seem 
to have enoueh time. 



5(a) The answer to this depends on how much wlfjht you 
attach to the word "particularly". The number '^liven 
(i.e. 10) were or would have been classed as "1" in 
your 1, 2 X syst«jm. 

it is my view that the service has not been or value 
owinty to the transient nature of my own interests. 
The majority or my Information problems have been con- 
cerned with literature searching new fields which 
will remain of importance tome for a limited period 
rather than keeping.: up-to-date on a smaller number of 
topxcs which are of sustained intex^est. Had my inter- 
ests been more specialised and lon^v term X am sure the 
servLce could have been of greater value. 

Thank you for allowln/>- me to participate In tills ex- 
ercise. I would welcome an opportunity to see an anal- 
ysis of tlie results. 

The present system would seem to be satisfactoi-y for 
people working' on particular subjects for some time. 

k62 

I have retained the cards givin;; references in three 
croups corresponding to the 1, 2 and X relevances. 
Observing the piles now confirms that there are very few 
cards very relevant to my main interest. lin retrospect 
my memory tells iiie that initially the service was poor, 
it beyan to improve and sent more relevant references 
to that main subject only after L had notified you of a 
chanfje required In my profile due to a chan/^e of occ- 
upation at work. I then continued to receive many (>ood 
references to a subj<.-ct which had been my prime interest, 
but which was now only of secondary interest. Only In 
the last month or two lias it appeared that the penny has 
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droppiwl iiiLo tUr coiiipii i.i- r pror;raiiiiii(? Llial, I liavr* cliaiif^ed 
my iiK.fMVj.sL. I !iav<- in'C(mv(»(I a fow roVovoncos on my' 
main suhJor.L of* inLor^sL ns mcnilJoncd only f'or aboiifc the 
la.sl. i-'irMl W(»(;k« whilsL my clianr/* of* inLor<;st. (.ook place 
ovor a yc:ar a^vo* Now I am clianr;iiwv my in Lores t a/vai.u so 
LhaL tho service would Ix* ou I; of* dal,o a^jain. 

In /joneral the sorvi.co i.s useful but wo have a strong; 
pressure to read all useful papers and these usually ' 
appear in journals In the company library so T only save 
the trouble of maki.n^v out my own Ti.le cards for userul 
papers • 

^168 

It must bo obvious tliat I am one of tlic probably very 
few who slipped entirely through your net. I have thou^^lit 
a creat deal about tli i s without being able to offer any' 
really satisfactory solution. i' work on a vide i^ange 
of microwave devices but my interests are almost entirely 
technolo/j'ical. Perhaps the types of periodical are too 
"academic" or "fundamental". Perhaps the type of article 
r need lacks appropriate coding words in the title. It 
does seem from remarks of collea/^ues and my owji items 
that you cater very much more for the research/develop- 
ment man than the practising engineer. 

Nevertheless, good luck and thanks for trying. 

I think my main comment lias already been made and Indi- 
cated in the answer to the last question, viz.: I would 
have been happior if during the course of the SDI Inves- 
tigation one^s profile had been modified by means of feed- 
back to ensure that a liigher proportion of relevant 
articles had been received. 

^70 

Q.I. A higher rating could have been applied if the 
service had been more regular and prompt. 

Q.2. Difficult to answer. We have certainly found 
many relevant articles about v/hich we were not 
notified. You may, however, not have scanned 
these particular journals. 

Q. Mainly due 'to incomplete literature coverage by 

yourselves and delay in receiving returns. 

Q. 8. Relevance returns should only bo made If* the user 
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Q. 10 



471 



is not saLi.sri.ccJ with tlic yield. 

A fortnightly service would bo acceptable if it 
covered articles published in the previous 2-3 
weeks . 



The servi,co was found to be useful but not essontLal* 
the omission of manufacturer's literature and advor-' 
tisements meant that 1 !iad to pursue many of the rel- 
evant puljiLcati ons anyway. 



As my field of i.ntorest is narrow this 
s i.ble . 



i.s eas Lly pos- 



r consider that the cost of continui-nfr the service is 
"y I^^^'sent duties i.t Ls not Justified by 



the time saved. 



I beJ/iovo the service niLssed an article of iiii.no pub- 
ILshod in Mullard TocJuiLcal Coiimiunications May 1969 
Vol. 10 No. 99 entitled "Four phase Lo{jic Cix^uLts 
usin/r MOS Transistors". 

hn 

It is too expensive for a private individual. Annual 
subscr.Lpt,Lons would probably have to be made by one's 
firm. 

481 

!.n creatLnc a prof Lie, it would have helped if a list 
of "standard" keywords were used. t think that the ar- 
ticles I mLssed were due iiiaLnly to ambi<ruitLes Ln these, 
a.!thou{yh these mitrjit have eventually been reduced by 
ex tend in/; the profile. 

482 

Hn the field of systems en/;ineer i ri<>:, review articles 
are often the most usel'ul ones in producLn/; new Ideas 
or ways of tackling.; a problem. In this field, ma/y- 
azines like Av.Lation Week are very /yood, yoL do noi 
seem to be covered by SDI . 

i: stLll do not feel tliat my profile is qu L to ri/yht, al- 
thou{rh \ have not done any thin/; very posUive to help 
apart from returning; tJie relevance assessments. 

ii2£ 

The immediate access to relevant data has proved valuable 



11-38 



I.n extomilnfi this system to my requirements of a ner- 
manont record of information sources, it would be use- 

r-1 ""^J^^^"^ '-'"^ ^^^'^^ "^-^"il keywords of my 

profile. This Ls difficult at present as \ have to 
photocopy cards wLtli more than one oJ' my keywords in 
or«J«.r that a simple systom could Ijo complete without 
<:ro.s.s rr.I,,ronces. The provision of additional copi.es 
uL'fCl keyword occurs would be 



'501 



1 tlii,nk It IS vronn to assume tliat casual roadin/- .s 
necessarily unproductive. ! tend to 
each week do in/-; so as 



spend some < 
form of relaxati.on. 



line 



Tho unforesoeii value of the service has been tlie oc-» 
cas.onal article on the 'od^jo' of the profile which was 
i.n lact ot unreco;;in.sod relevance. The debit side is 
the numb.r of articles which were scanned, when copies 
were obta.ined, and were not of interest. The latter 
article i.s many cases could not be Judged by their titles, 
However, overall, there is no doubt that some very in- 
terest. n^r articles did come to light which might other- 
wise have been missed - hence tho increase in reading 



As per my previous letters. 
516 



.nvL!f"?T ""f!! .'^f '"''^ diversity of interests such as 
"•ysclf L I car that by scanning Current Papers 1 pick up 
a lot of articles that are not notified by SDT. t am not 
clear whether this is because a) all articles in current 
papers may not be scanned by SDI, or b) „,y profile 

ih? '''''' keywording is inadequate. Wliilst 

Sbl. ..s useful as an a.Jditional source, it is a long way 
from being a satisfactory alternati.ve to Current pipers - 
and t still consider it will be essential to scan them 
lor a long time to come. 

Whilst not wishing to l)elittle the SDI experiment and 
luily appreci-ating tliat it is only intended at the 
moment as a current awareness system 1 feel that the 
usefulness of computer sorti.ng of information may havo 
greater value in the field of information retrio vm. 

u7'ilo I'T-' >'-^^ ' i'ave had people taking 

up the study of new r«-.sc>arch topics and the time spent 
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suhsV^H Kr "■'--'."'-A''s'-.-acU. oLc. ,.as boon 

i ir i';; w«uhl have boon ntalu! 

-abio. ! Jicixjwo may l.avo one one (Jay. 



our'aitfv?tics'ira'^:"'-°'' ''<^'''"'"'^ cloarly the area of 

:::t::-':;r"?;v-" ? ^^^^^^^^^^ 

ohuor concern. Wo cannot completely exclude J^r, 
a= contacs, injection, MOS, fi,:s, eLfSsrff,„t":3eence 
S:enrT.ar??c^:;tr.;ai;^?l:t:°/'-'- 3urr2::r!nUr. 

Ultimately the pracUco of scannln,,; journal artLcles 
,1L^ by eye - chock in,.- umnod Lately on key words bv 
reference to the abstract and the text is L ?^e cfr 
umstanax..s the onJy Pool-proof method. ' 

LrllT^ r"'' probably excellent for a well 

?v[jy otc!^'^ '"^'•"'-^^^^ .naterials - superconduct- 

l would be Lnterested to know Lf you can see any possi- 

a.iJ'.iLr.:ri:-^n°'t.:n ^-tesr- ::;i?'>tr- 

work"" Howelf "fh''""''"? <^-tremoly valuable aid to n.y 
work. However, the simultaneous improvement that has 

h^r madfth^ as" ^^^^^^ ' ^"^ ' Current' AistJJcts • 

eJLnt?L Thr.? tailored service less 

essentxai. The class .ficat Lon scheme adopted Is suf- 

moMUr r\r'''r '° ^""^^^ ^ ^^^^^^ nado eJci 

nio.ith of the relevant literature. It also allows for 

thir.irbr''^'"'^'' or random browsing, In^^lZL^l 
fum^l on c Inefficient, it performs the 

function of mtroduci n.- random mutations into ones know 
led,.;e, thus allowing; a procoss of evolution. " 

t-l^'-^^^T^ "'t!^ ''^"^'^ '^'-^ valuable would be the 



slbility of literature searche;: n;;efull. iSis wil^be 
?rf' ^'i?..^^ i-ncrease your data base. J ^oufd sav 

that off uc.ent searches would be worth some lens of pounds 
per search on avera,,e, and this probJom is often of ?t"afoi 
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iiiiportanc(i than that oV updatln^^' current Lnformation. 

In conclusion I would like to say how much I reel pri- 
viJ.o^^od to have boon associated with your project, and 
wi,sh you success in oxtendin/v your work in the future. 

520 

I feel that any shor tcoiii i.nr;s In my no t i.f Lcati ons are 
largely duo to my lack ol' t.ime to take pcsiti.ve steps 
to react. I liopo to r.ive this matter more attention i.n 
the f*u ture . 

Please accept my tlianks and appreciation for vour very 
co-operative and lielpl^ul activity. 

521 

I find the SDi service extremely useful. The main pro- 
blem L liave is that frequently there is a considerable 
delay between recoivii\vthe SDI cards, and the approp- 
riate journal bein/j received by our library, for 
American journals. I have no complaints about tlie SDI 
service i tself . 

222 

It seemed to me that T received far too many irrelevant 
articles of little Interest - which T feel was due to 
the fact that the profile did not enabj 3 me to narrow 
down the field sufficiently for my interests. The field 
of Gas dischargees is very wide and the number of words 
available for the profile was too limited. I also find 
that many oV the articles of interest to me, are In 
journals such as Electronics, written by editors etc. 
and none of these are picked up by SDI. 

328 

To date the service has provided the most comprehensive 
and clear cut metliod of obtainin^j references that I have 
pers onally encountered . 

Thank you for the orl^vlnal offer to participate. 

m 

C think tlie service made a very ;;ood job of i n terpro tin^; 
the profile which I. wrote. With a wide Ta.nr:inr: prof^lle 
there must be many notifications of minor interest but 
I accept this as Inevitable. 

Do you store the i.nformatlon In a form such that notlfl- 



1 



if 



cations on a parLLcuiai- subject which have been issued 
over the past .1 or 2 years can bo supplied on demand? 
r think tJiLs uould bo a very useful service to those 
embark i.n{; on a new field. 

Sti.v<ia?a;i; .-a-rlLclos of iiitorost to us arc Ln journals such 
as J. CiioiiiLcal Physi.cs whLch arc not covered by your 
service. 

532 

It appears that my pronie has failed to pick out some 
uiiportant articles - others have not been notified as 
they appeared in periodicals not covered by your scheme. 
For .example, E have no record o£ bein;^ notified of 
three mass spectrometry papers in 'Vacuum' vol. I9, no". 7 
bj McCraoken, Ball and Wilson resp. 

T. have recently been us ln{; 'Mass Spectrometry Bulletin' 
published by AWRE, Aldermaston. Although most of Its 
references are of no relevance, f. have found those which 
are to be more comprehensive than those noti.fied by the 
SDT service. 

Cncldentally, E did ask for dielectric aerials to be 
Included In my profile, althou^-rh it was not now of 
relevance in my work. Several articles on this subject 
have appeared in ' Electronics Letters • and none have been 
notified. 

(1) There is a fair proportion of reference, which are 
entirely irrelevant to my profile. Some of these are due 
to the rather open ended nature of the profile in certain 
areas but some are inexplicable to me. In the example ,[ 
enclose E cannot see the relevance of many of the key 
words to my profile. Speaking-- more generally It is a 
basic weakness of the system that the same quite long 
list of keywords can still pick out 2 activities which 

.In practice are virtually unrelated. This weakness is 
sometimes rather apparent. 

(2) Despite (1) above I. am quite happy with a system 
which throws up many Irrelevant Items - it doesn't 
take me long to dispose of thenii Lt -Is much more 
serious If relevant articles are missed, Ln this 
connection J wo\ilfl welcome a mucli more complete coverage 
of a) conference proceedings and b) Government reports 
(particularly U.S.A.). Perhaps you could pay more 
attention to discovering what conferences are being 
held and making sure that all proceedings are Included 
in your system. 
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536 

Dos i do Lho possibilihy oV soTidinr; i.nrormat Ion by direct 
accoss Lo a r;raphlc cJLsplay LeriiiKxial, a facLlity most 
lar^i'eorr;ani. sat ions will, shortly acquLre, It would be 
benefLci.al to be able to call up references in other 
fields, not connected to tliose in the profile, without 
modifying' the existing; profile* This would only be of 
occassional use, but iiiLcht stimulate cross-fertilisation 
of ideas. 



This informatLon would only be required occasionally, but 
could determine whether the system was compe t i. t Lve. 

A very lar^^e number or cate^^ory (2) cards were received 
in relation to (l) and (x). This may have been partly 
die to difficulties in defining my profile. I feel it 
would have helped if subjects had been classified 
according to, say, the Universal Decimal Classification 
for the purpose of the investigation. 

I should like to thank you for all the helpful infor- 
mation you have sent me during the past year. 



I found the service most successful when applied to 
specific topics e.g. J .F.* amplifiers or Phase Discrim- 
inators. My profile, which was based on my systems 
interests, was really too generalized to be of ^^reat value 
however, I only discovered this after about six months 
or so. Another problem is that ones interests change 
with time and this might require radical alterations to 
the profile. 



A generally useful service, but it overlaps a similar^ 
service from our works library*. 

The service soems to liave achieved Its objectives, however 
in my case it would bo fairly safe to limit the search to 
about three periodicals for highJy relevant material and 
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perhaps 20 more for less relevant background topics. 



5k7 

The SI).L service is potentially very useful. However, 
we have an excellent 1 i.brary abstract service on site 
which (jlves a much lar^^er selection than does your 
service. Special articles can easily be obtained from 
Barnard Castle. 

This service is quite essential in my view whei^e local 
(works etc.) library servico? aro inadequate, therefore, 
in my view the costs oC services even at £35 were not 
too much but in niy own case your service was not so 
essential as our library service is very comprehensive 
indeed. 

Thank you very much for the efforts you and your staff 
had put in this inves I i /Ration for the benefit of industry 
as a whole. 

Hi 

A broad profile may result in a relatively hi{j;h proportion 
of irrelevant items. i: believe that the SDl service will 
be more useful when profiles are narrow and well defined. 
For more varied requirements reference should be made to 
a summary of abstracts before articles are selected. 

The selection of short abstracts of articles. Instead 
of the selection of tlie titles of such articles, would 
considerably increase the value of the service. 

559 

We have a very (jood and speedy library service.. They 
produce a very ^ood weekly abstract on library additions 
thus short-circuitin(^ your service on rece jved journals. 
Hence our requirement Is for journals, such as forei/jn 
publications not taken, to be dealt with by your. 
Thus as far as I am concerned you have been scanning all 
the wron(:j, i.e. popular, journals J 

If you scan more obscure journals then it could be of use 
hut will eliminate completely the need for things like 
Current Papers. 

I have therefore, recoirmiended that our library subscribe 
on a Group basis but with their intake of journals in- 
hibited. This should eleiminate 90^ of possible cards 
and provide us with usually obscure information. 



ERLC 



11.44 



fJoiKjrally tho 3.ocal lilnary providos a r.ood service 
cover the journals uva liable in the library. A few 
journals not covered liuve had usoTul papers in them 
which were picked up by SDI. Usefulness is limited 
(s^e opposite) because of security classification requir- 
ing special communication publications in some fields in 
which I am interested. 

Also about half my activity Is concerned with forward 
projects and new applications, where the subjects are 
broad and fast changlii/y but probably not having, for 
various reasons, much current publication. Consequently 
1 have not updated my profile to attempt to cover these. 
However, on the topics requiring my more sustained 
activity 1 had come to fall back on SDI as bringing 
articles to my notice, in conjunction with our local 
library service, rather than spending time scanning. 

1 am a development engineer in a- coiranercial firm. I 
find that for, say, 1 month in a year 1 am actually 
•designing'. For the other months I am testing, 
engineering, dealing with factories, with faults and 
comments from users etc. 

During the 1 'design* month I want access to information 
on many different aspects of the job. In this I find the 
local library index. Science Abstracts and my own private 
notes very useful. Also manufacturers' publications are 
invaluable. 

For the rest of the time I don't want to be bothered with 
reading. If 1 see a good relevant articles, all I do 
is to make a note of It. The time for incorporating any 
good ideas is past once the design stage is over. 

As a suggestion, it might be useful, if for a fee, one 
could ask for (say) 'List of all articles published durin 
the past 2 years on voltage-control.l ed attenuators for 
frequencies from 3-2- MHz. ' But would this be any 
quicker or more thorqugh than looking through Science 
Abstracts? I doubt it. 

I found the service a useful adjunct to our internal 
library services which picked out a small number of 
items I would otherwise have missed. 

My personal feeling is that though the service could be 
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improved and select all articles of direct relevance 
one would still havo to scan a lar^^e volume or literature 
in order to pick up useful ideas, techniques and approaches 
used i.n other fields. For instance a technique for can- 
celling; hum pick up if? a blo-medical instrumc-nt could be 
used in a TV camera. A computer will never pick out this 
sort of thing from scanning an eight or ten key word 
summary. 

L thlnlc the main value of this inves ti /vation is that a 
large number of journals are systematically searched. If 
one searches oneself it is time consuming and It is very 
easy to miss particular issues. 

I find that I need more time to myself in order to study 
the papers and books upon which 1 lay hands rather than 
yet more articles from an ever increasing number of 
sources. 

In industry one must give the impression that one is 
actively working upon the job in hand] Sitting and read~ 
ing gives the impression to managers that one Is slacking 
or too tired to workJ Therefore, I read mainO^y at home.'' 
At home, I find that in order to ''make ends meet" I 
am obliged to spend many hours in decorating, digging 
the garden, mending the car, 'mending the telly,, etc If 
I had sufficient salary to pay for this work to be done, 
I would gladly spend the time thus gained upon more read- 
ing and thus making myself more efficient at my work. 
Judging by the 1.5 million people (mainly skilled workers 
and their families) who have left this country during 
the past 5 years I. guess that many people feel the same as 
I do. 

After giving considerable thought to these problems I 
believe that the only way to have a satisfactory career 
in engineering under conditions obtaining in this country 
is to stay single throughout. (Age A?). 

The number of irrelevant articles notified reveals a 
weakness in the method of profile definition. This may 
well be the fault of tlie participant, erg* a Tar too 
general interest profile Is given Initially for fear of 
limiting the suefulness of the service. 

It would help if it could be suggested that It may be in 
the interest of the participant to limit his field of 
interest for the initial profile. 
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My present Informal J on requirement Is for articles coveri 
the subject of F.D.M. Carrier Telephony in fairly general 
terms. As I don^t need to pursue every related subject i 
considerable detail i feel I am unable to make the best 
possible use of the SDI Service. 

I thought the relevance of the material improved as the 
exercise continued. 



58k 

I am perhaps not a typical example of the type of worker 
who would need the SDI service* I make it my business 
to at least scan the cojitcnts lists of most of t)^e 
"standard" En^^lish, Aincrlcan and Russian publications, 
and am assisted In th by the exceptional facilities 
available to Marconi O. Baddow Library. For my own needs, 
I shall prefer SDi to provide more Information: a) from 
the more obsecure f ore L^jn journals e.g. Czech, Hungarian, 
Chinese. b) from tlic theses published by European, 
American and Russlon Universities. I have great difficulty 
in finding out what is available in these areas. 

1§1 



The class of user who definately derives most benefit 
from INSPEC SDI is the research student, and personnel 
engaged in base level investigations as opposed to senior 
mana ge m e n t pe r s onne 1 . 

Whilst executive personnel could offord the service those 
at lower level certainly could not. I was extremely sur- 
prised when I discovered the cost of retaining SDI mem- 
bership - completely out of all proportion to the value 
it gives me^ 



587 ' 

The one comment I would make is obviously a product of 
my special circumstances (in that no funds were available 
to me and in that my particular requirements arc found in 
but a few journals); but the conmient may be relevant if 
others are in a similar position "in number". 

The vast majority of the articles I have found useful 
and interesting occurs, roughly evenly distributed, in 
about ^k journals; all but 3 of these are taken by our 
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library h'»rc5« The (*f>,s(. oP Ihr. prc^sont SDI service per 
aiiiiiiiji wouici prohahly rovav IJm* cosI. of subscr i.p t i.on to 
IJidSo '3 journals. Thr i ii forma i i on I would not come 
across IMMEDIATELY If I then merely used these 14 jour- 
nals alone would be, at an estimate, 5^ of my total. 
Although (judging: by the accuracy of your service) T 
v/ould not miss the 5^ by subscribing to SDI, the extra 
convenience of having the extra 3 journals would, I 
think, be a worthwhile trade for it • 



.589 

As mentioned abov<» I am 'at present in a sLtuati.on where 
X am unable to s tucJy tlio Literature to the extent that I 
would normally. For tho purposes of the SDK Investigation, 
therefore, J have interpreted "relevant" in the sense 
of relevant to the field in which I would like to be 
working if T. were able to choose • 



The proportion of hi,ghly relevant references was great- 
est right at the start, and has shown a gradual decline. 
At the same time, the number of non-relevant references 
which was small at first, have gradually increased, 
reaching a peak at about two thirds of the way through 
the investigation. The number c?f partially relevant 
refs has increased noticeably, and this represents some 
degree of success. ^ 

It looks as If the method of feedback has not woi^ked 
as well as it might have done. Ht could be, though, 
that these trends are not significant because the 
sample is not big enough. You will no doubt lump 
them all together for analysis and come up with a more 
confident answer. ^ 

704 



Although SDI has been shown capable of meetinci* our 
specific needs I feel that the individual research work- - 
er still benefits from scanning the contents of journals 
regularly in order to see what is going on outside Ixis 
speciality. Ef SDI (or a similar scheme) was the only 
Information service we would become extremely narrow 
in outlook- The main value of SDI has therefore been 
to back up the ordinary library service rather than to 
replace i t . 



701 

£ support SDJ wholeheartedly In principle and J.ook f'or 
ward to refinements as proposed under (2). 
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712 

(1) Undor item 5 the; <liscropency between the numbers is 
duo we Vital to our proPLle, which we did not change too 
drastically because wo were interested in testing the 
system as it stood^ Also the nuiiibor of "highly relevant" 
returns which we made would not, of course, represent the 
number of papers which actually helped us with our 
research. 

(2) The answer to 3a is due to the greater attention now 
paid to infoimation retrieval. 

(3) We have been very Interested to take part in this 
survey. 

ni 

As indicated over leaf K feel that insufficient guidance 
was given on the basis for the SDl and profile defin- 
ition. This may be an unwarranted criticism. However, 
we now are in the process of updating our profile which 
should result in a better service. 

Tjjt 

One has still to reniomber to look at the cards and they 
should also be filed i.n same order. Both time consuming 
and not thought worthy of the effort. 

221 

The main difficulty is reading the articles uncovered which 
is hardly your problem. 

716 

With the co-operation that is supposed to exist between 
the lEE and the IEEE we ought to be able to reduce the 
delays in obtaining TEEE publications . 

The most up-to-date work is often presented at confer- 
ences, then later published after a long delay. It 
would be useful to know Of work presented at conferences, 
without waiting for the conference publication to appear. 

721 - • 

Like most RAD personnel, the writer has for many years 
kept a personal reference system, by subject headings 
.e.g. Amplifiers, audio, Pulse circuits. Modulation and 
demodulation etc. This uses standard index cards. It 
would have been helpful if the SDI cards were of the same 
size, to enable integration into this system. 
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726 



My o i rcums lano<5S arc pi obahJy poouLiar In tliat T have 
niovcsd away fVoiii tlu? particular f*iel<I of Ln tores t - 
'e J.<scti*onics ' - duri.nr, Uio perLod of (;lie investigation. 
I have retained an acfxdoiiiic i.nLerost Ln the topics 
represented by my prolM.Je whilst in practice I have had 
no chance to follow any of the work Ln the scientic 
literature. As a result niy readLn^.^ and follow up of the 
SDI performance has b(»on zero. V marked SDI notifications 
as relevant or otherwise Ln the spLrLt of the profile and 
many pap^^rs of potential interest liavc i.ndeed been 
notifi.ed to mo. However, Lt is apparent that I have not 
fully co-operated and cannot gLvo moanin{jful replies to 
many of the precedLn{j questions. 

Tn my present occupation I g^neraily need information 
which is not new and is outsLde the scope of the in- 
vest ifjation. There is therefore, no poLnt Ln subscri- 
bin(^ to the continued service in the immediate future. 
I shall of course consider re-jo Inin^^v when my company 
starts serious research and development work. 

728 

T reals Le now that the pronie needed updating, in 
particular "Signal processing, with particular reference 
to correlated and optical me.thods" - the subject is much 
too broadly defined. 

I also found that some references on electro-acoustic 
tran:5ducers for U.W. sound slipped through the net. But 
certainly some usoful sources and references came to 
light. 

no 

I think Lt would be mofjt interesting to see how the SDI 
staff have modified our profiles to yied better res- 
ults. 

732 

Thank you very much for your service over the past months. 
The only useful comment I wish to add is that i t would be 
very nice to change ones profile quickly to a new topic 
of interest, and have confidence of getting.?, it riglit 
within say 1 montli. 1 realise this requirement is 
probably not general. 

I thinJc that the system is a good idea. I do not always 
give the cards my immediate attention and I have not 
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always looked up the papers which X think that I should 
have, but certainly r have been iiiucli more aware of what 
has been publLshod tliat I would have been without ttie 
cards. 

May I repeat that the cards would be oven more useful 
i.f each carried an abstract of the article referred to. 

Thank you for i.ncludi.n/7 SCW i.n your system. 1. have 
found i.t useful, liopo you havo cai.nod also from our 
part ic i pa t i.on, 
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My only real intorosl.s concox^ns Llic lar^e number of 
rolcvanL arti.clcs MnL.ssod' by I.NSPEC. 

1 bciiove (;ha(; Lliis coulcj be Improved Li' a far more 
detaiiod description ol' my x^oqu i.rcMnon(;s had been asked 
for at: the be{jiiuu.n{^, 

ExLsCin/v Company Li.b.rary costs cover literature surveys 
and are self sufflci.ejit for my needs as well as bein/r 
more specific. 



